The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the ...
The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the ...
Gain familiarity with existing literature on data annotation and LLM as judge * Understand NIST's role and ongoing efforts in assessing and measuring the validity and reliability of AI‑related ...
Gain familiarity with existing literature on data annotation and LLM as judge * Understand NIST's role and ongoing efforts in assessing and measuring the validity and reliability of AI‑related ...
Artificial Intelligence / Machine Learning Lead Verification and Validation Engineer
Reston, VA · On-site +1
$106K - $140K/yr
Establish and evolve annotation standards, labeling guidelines, data governance frameworks, and quality assurance processes to ensure consistency and reproducibility at scale * Design and oversee ...
Artificial Intelligence / Machine Learning Lead Verification and Validation Engineer
Reston, VA · On-site +1
$106K - $140K/yr
Establish and evolve annotation standards, labeling guidelines, data governance frameworks, and quality assurance processes to ensure consistency and reproducibility at scale * Design and oversee ...
Artificial Intelligence / Machine Learning Lead Verification and Validation Engineer
Reston, VA · On-site
$106K - $140K/yr
Establish and evolve annotation standards, labeling guidelines, data governance frameworks, and quality assurance processes to ensure consistency and reproducibility at scale * Design and oversee ...
Artificial Intelligence / Machine Learning Lead Verification and Validation Engineer
Reston, VA · On-site
$106K - $140K/yr
Establish and evolve annotation standards, labeling guidelines, data governance frameworks, and quality assurance processes to ensure consistency and reproducibility at scale * Design and oversee ...
Paralegal
Fairfax, VA · On-site
$125/hr
Provide structured written feedback on complex legal content to support AI training, data annotation, and quality assurance processes. * Evaluate complex legal materials and apply strong judgment and ...
Paralegal
Fairfax, VA · On-site
$125/hr
Provide structured written feedback on complex legal content to support AI training, data annotation, and quality assurance processes. * Evaluate complex legal materials and apply strong judgment and ...
Artificial Intelligence / Machine Learning Verification and Validation Engineer
Reston, VA · On-site +1
Develop and maintain annotation standards, labeling guidelines, and data governance processes to ensure consistency and reproducibility across projects * Conduct IV&V of datasets and model outputs to ...
Artificial Intelligence / Machine Learning Verification and Validation Engineer
Reston, VA · On-site +1
Develop and maintain annotation standards, labeling guidelines, and data governance processes to ensure consistency and reproducibility across projects * Conduct IV&V of datasets and model outputs to ...
Develop and maintain annotation standards, labeling guidelines, and data governance processes to ensure consistency and reproducibility across projects * Conduct IV&V of datasets and model outputs to ...
Develop and maintain annotation standards, labeling guidelines, and data governance processes to ensure consistency and reproducibility across projects * Conduct IV&V of datasets and model outputs to ...
Develop and maintain annotation standards, labeling guidelines, and data governance processes to ensure consistency and reproducibility across projects * Conduct IV&V of datasets and model outputs to ...
Develop and maintain annotation standards, labeling guidelines, and data governance processes to ensure consistency and reproducibility across projects * Conduct IV&V of datasets and model outputs to ...
Buscas una forma flexible de generar ingresos extra? Estamos invitando personas a participar en un proyecto de entrenamiento de IA grabando actividades reales de su dia a dia desde casa. Graba cuando ...
Buscas una forma flexible de generar ingresos extra? Estamos invitando personas a participar en un proyecto de entrenamiento de IA grabando actividades reales de su dia a dia desde casa. Graba cuando ...
Gana dinero grabando actividades de tu vida diaria Buscamos personas para grabar actividades cotidianas desde una perspectiva en primera persona utilizando una aplicacion movil. Puedes grabar tareas ...
Gana dinero grabando actividades de tu vida diaria Buscamos personas para grabar actividades cotidianas desde una perspectiva en primera persona utilizando una aplicacion movil. Puedes grabar tareas ...
English-Spanish Bilingual Speaker - Voice Recording (Remote)
Herndon, VA · Remote
$11.25 - $12.75/hr
This is a project-based opportunity on an AI training platform - not a job. No fixed hours, no commitment beyond what fits your schedule. You talk, you get paid. About the Role We're looking for ...
English-Spanish Bilingual Speaker - Voice Recording (Remote)
Herndon, VA · Remote
$11.25 - $12.75/hr
This is a project-based opportunity on an AI training platform - not a job. No fixed hours, no commitment beyond what fits your schedule. You talk, you get paid. About the Role We're looking for ...
Gana dinero grabando actividades de tu vida diaria Buscamos personas para grabar actividades cotidianas desde una perspectiva en primera persona utilizando una aplicacion movil. Puedes grabar tareas ...
Gana dinero grabando actividades de tu vida diaria Buscamos personas para grabar actividades cotidianas desde una perspectiva en primera persona utilizando una aplicacion movil. Puedes grabar tareas ...
Buscas una forma flexible de generar ingresos extra? Estamos invitando personas a participar en un proyecto de entrenamiento de IA grabando actividades reales de su dia a dia desde casa. Graba cuando ...
Buscas una forma flexible de generar ingresos extra? Estamos invitando personas a participar en un proyecto de entrenamiento de IA grabando actividades reales de su dia a dia desde casa. Graba cuando ...
English-Spanish Bilingual Speaker - Voice Recording (Remote)
Herndon, VA · Remote
$11.25 - $12.75/hr
This is a project-based opportunity on an AI training platform - not a job. No fixed hours, no commitment beyond what fits your schedule. You talk, you get paid. About the Role We're looking for ...
English-Spanish Bilingual Speaker - Voice Recording (Remote)
Herndon, VA · Remote
$11.25 - $12.75/hr
This is a project-based opportunity on an AI training platform - not a job. No fixed hours, no commitment beyond what fits your schedule. You talk, you get paid. About the Role We're looking for ...
... data / annotation processes to enrich unstructured data leveraging a variety of data sources (social media, news, internal / external documents, images, video, voice, emails, financial data, and ...
... data / annotation processes to enrich unstructured data leveraging a variety of data sources (social media, news, internal / external documents, images, video, voice, emails, financial data, and ...
... data / annotation processes to enrich unstructured data leveraging a variety of data sources (social media, news, internal / external documents, images, video, voice, emails, financial data, and ...
... data / annotation processes to enrich unstructured data leveraging a variety of data sources (social media, news, internal / external documents, images, video, voice, emails, financial data, and ...
Data Scientist
Ashburn, VA · On-site
$99K - $164K/yr
Support exploratory data analysis, feature development, image-data preparation, annotation, training-set construction, model evaluation, and model-improvement activities. * Assess model performance ...
Data Scientist
Ashburn, VA · On-site
$99K - $164K/yr
Support exploratory data analysis, feature development, image-data preparation, annotation, training-set construction, model evaluation, and model-improvement activities. * Assess model performance ...
Data Scientist
Ashburn, VA · On-site
$99K - $164K/yr
Support exploratory data analysis, feature development, image-data preparation, annotation, training-set construction, model evaluation, and model-improvement activities.Assess model performance ...
Quick apply
Data Scientist
Ashburn, VA · On-site
$99K - $164K/yr
Support exploratory data analysis, feature development, image-data preparation, annotation, training-set construction, model evaluation, and model-improvement activities.Assess model performance ...
Data Scientist
Ashburn, VA · On-site
Support exploratory data analysis, feature development, image-data preparation, annotation, training-set construction, model evaluation, and model-improvement activities. * Assess model performance ...
Data Scientist
Ashburn, VA · On-site
Support exploratory data analysis, feature development, image-data preparation, annotation, training-set construction, model evaluation, and model-improvement activities. * Assess model performance ...
Data Processor
Vienna, VA · On-site
They are seeking a Data Processor to support AI Engineers by preparing, annotating, and validating ... with image annotation tools or similar platforms. • Ability to identify cracks, pavement ...
Data Processor
Vienna, VA · On-site
They are seeking a Data Processor to support AI Engineers by preparing, annotating, and validating ... with image annotation tools or similar platforms. • Ability to identify cracks, pavement ...
Data Annotation information
What is a data annotation?
A Data Annotation job involves labeling and categorizing data, such as text, images, audio, or video, to help train machine learning models. Annotators apply tags, bounding boxes, or classifications to data based on specific guidelines. This process improves the accuracy of AI systems in recognizing patterns and making predictions. Many data annotation jobs require attention to detail and familiarity with specific domains. It is commonly used in applications like autonomous driving, natural language processing, and computer vision.
What does a data annotation do?
A typical workday as a Data Annotator involves reviewing datasets—such as images, audio, text, or video—and accurately labeling or categorizing information according to specific project guidelines. Most Data Annotators work independently, but they often collaborate with project managers or data scientists to clarify requirements and resolve ambiguities. Tasks may be repetitive, but adhering to precise standards is vital for maintaining data quality. Work environments can range from technology companies to remote or freelance settings, and advancement opportunities exist as team leads or quality assurance specialists for those who excel in consistency and reliability.
What are the key skills and qualifications needed to thrive in data annotation?
To thrive in Data Annotation, you need strong attention to detail, accuracy, and basic data handling skills, often supported by a high school diploma or equivalent. Familiarity with annotation platforms, data labeling software, or content management systems is frequently required, though specific certifications are rare. Excellent communication, time management, and the ability to focus on repetitive tasks distinguish top performers in this role. These skills are crucial because accurate and consistent data annotation directly impacts the quality of machine learning models and AI applications.
How much money can I make doing data annotation?
What are the most commonly searched types of Data Annotation jobs in Leesburg, VA?
The most popular types of Data Annotation jobs in Leesburg, VA are:
What job categories do people searching Data Annotation jobs in Leesburg, VA look for?
The top searched job categories for Data Annotation jobs in Leesburg, VA are:
What cities near Leesburg, VA are hiring for Data Annotation jobs?
Cities near Leesburg, VA with the most Data Annotation job openings:

Assistant Research Scientist (PREP0004176)
Gaithersburg, MD • On-site
Full-time
Re-posted 6 days ago
Johns Hopkins University rating
8.0
Based on 71 frontline employees who took The Breakroom Quiz
Job description
PREP Research Associate
This position is part of the National Institute of Standards and Technology (NIST) Professional Research Experience Program (PREP). NIST recognizes that its research staff may want to collaborate with researchers at academic institutions on specific projects of mutual interest and, therefore, requires those institutions to be recipients of a PREP award. The PREP program involves staff from a wide range of backgrounds conducting scientific research across various fields. Individuals in this position will perform technical work supporting the collaboration's scientific research.
Research Title:
Reliability of Human and LLM Annotations for AI Risk Assessment
The work will entail:
This project focuses on using Large Language Models (LLMs) to provide annotations of evaluation data (a.k.a., LLM as judge), and the design of an Inter-Annotator Agreement study to assess the reliability of both human and LLM annotations. The candidate will explore assessing the indicators of a given AI-related risk, determining how to identify them, and providing annotators with examples to annotate the presence of various risks. The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the validity and reliability of AI-related risks in data annotation.
U.S. Citizen Preferred
Key responsibilities will include but are not limited to:
- Gain familiarity with existing literature on data annotation and LLM as judge
- Understand NIST's role and ongoing efforts in assessing and measuring the validity and reliability of AI-related risks in data annotation
- Contribute to developing an annotation framework for AI risk assessment
- Collaborate effectively with cross-functional and interdisciplinary stakeholders to ensure successful project outcomes
Deliverables
- Contributions to a NIST report that supports ongoing NIST AI evaluation efforts focused on the design of an Inter-Annotator Agreement to assess the reliability of both human and LLM annotations.
Qualifications
- Background in Computer Science, Data Science, or related field.
- Education level: Bachelor's or Graduate Degree
- Strong interest in data annotation and AI risks
- Familiarity with scientific reading and technical writing
Application Instructions
Please upload the following with your application:
• CV/Resume
*Please limit C.V to 3 pages only and ONLY include a valid email address for your contact info. Your resume will not be considered if the following information is included on your CV/resume.
• Self portraits
• Phone number
• Home address/Country
• Citizenship status
• Languages spoken
• Sex/Gender
Privacy Act Statement
Authority: 15 U.S.C. § 278g-1(e)(1) and (e)(3) and 15 U.S.C. § 272(b) and (c)
Purpose: The National Institute for Standards and Technology (NIST) hosts the Professional Research Experience Program (PREP) which is designed to provide valuable laboratory experience and financial assistance to undergraduates, post-bachelor's degree holders, graduate students, master's degree holders, postdocs, and faculty.
PREP is a 5-year cooperative agreement between NIST laboratories and participating PREP Universities to establish a collaborative research relationship between NIST and U.S. institutions of higher education in the following disciplines including (but may not be limited to) biochemistry, biological sciences, chemistry, computer science, engineering, electronics, materials science, mathematics, nanoscale science, neutron science, physical science, physics, and statistics. This collection of information is needed to facilitate administrative functions of the PREP Program.
Routine Uses: NIST will use the information collected to perform the requisite reviews of the applications to determine eligibility, and to meet programmatic requirements. Disclosure of this information is also subject to all the published routine uses as identified in the Privacy Act System of Records Notices: NIST-1: NIST Associates.
Disclosure: Furnishing this information is voluntary. When you submit the form, you are indicating your voluntary consent for NIST to use of the information you submit for the purpose stated.
What Johns Hopkins University employees say
Pay
Benefits
Hours and flexibility
Workplace
Get the full story on Breakroom
About Johns Hopkins University
Sourced by ZipRecruiter
Gilman believed that teaching and research go hand in hand—that success in one depends on success in the other—and that a modern university must do both well. He also believed that sharing our knowledge and discoveries would help make the world a better place. In 145 years, we haven’t strayed from that vision. This is still a destination for excellent, ambitious scholars and a world leader in teaching and research. Distinguished professors mentor students in the arts and music, humanities, social and natural sciences, engineering, international studies, education, business, and the health professions. Those same faculty members, along with their colleagues at the university’s Applied Physics Laboratory, have made us the nation’s leader in federal research and development funding every year since 1979. That’s a fitting distinction for America’s first research university, a place that has revolutionized higher education in the U.S. and continues to bring knowledge and discoveries to the world.
Industry
Colleges, universities, and professional schools
Company size
10,000+ Employees
Headquarters location
Baltimore, MD, US
Year founded
1876