The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the ...
The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the ...
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
QA / Evaluation Lead
$45 - $50/hr
... annotation to senior-annotator adjudication * Validate quality scoring and IAA computation within the Innodata data layer * Support AI Solutions Engineer on evaluation design for SAM 2 and Frontier ...
QA / Evaluation Lead
$45 - $50/hr
... annotation to senior-annotator adjudication * Validate quality scoring and IAA computation within the Innodata data layer * Support AI Solutions Engineer on evaluation design for SAM 2 and Frontier ...
Project Perseus \u007C Speech & Voice AI Analyst - Polish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst - Polish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Project Perseus \u007C Speech & Voice AI Analyst - Czech Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst - Czech Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Project Perseus \u007C Speech & Voice AI Analyst- Swedish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst- Swedish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Project Perseus \u007C Speech & Voice AI Analyst - Finnish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst - Finnish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Project Perseus \u007C Speech & Voice AI Analyst - Spanish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst - Spanish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Project Perseus \u007C Speech & Voice AI Analyst - Greek Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst - Greek Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Project Perseus \u007C Speech & Voice AI Analyst - Italian Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst - Italian Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Project Perseus \u007C Speech & Voice AI Analyst - Danish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst - Danish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Project Perseus \u007C Speech & Voice AI Analyst - Russian Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst - Russian Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Project Perseus \u007C Speech & Voice AI Analyst - Arabic Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst - Arabic Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Project Perseus \u007C Speech & Voice AI Analyst - Turkish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Project Perseus \u007C Speech & Voice AI Analyst - Turkish Speakers
Washington, DC · On-site
$26 - $28/hr
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
... power AI systems by executing data labeling tasks while ensuring accuracy and consistency. Responsibilities : • Execute high-volume data labeling and annotation tasks across speech and voice ...
... power AI systems by executing data labeling tasks while ensuring accuracy and consistency. Responsibilities : • Execute high-volume data labeling and annotation tasks across speech and voice ...
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Quick apply
Execute high-volume data labeling and annotation tasks across speech and voice datasets * Follow ... Support baseline data production needs for AI training pipelines * Contribute to team calibrations ...
Ai Annotation information
See Reston, VA salary details
$104.9K - $112K
18% of jobs
$114.5K is the 25th percentile. Wages below this are outliers.
$112K - $119.1K
21% of jobs
The median wage is $125.1K / yr.
$119.1K - $126.1K
14% of jobs
$126.1K - $133.2K
5% of jobs
$133.2K - $140.3K
4% of jobs
$140.3K - $147.4K
4% of jobs
$147.4K - $154.5K
4% of jobs
$154.5K - $161.5K
4% of jobs
$163.3K is the 75th percentile. Wages above this are outliers.
$161.5K - $168.6K
4% of jobs
$168.6K - $175.7K
10% of jobs
$175.7K - $182.8K
11% of jobs
$104.9K
$138.9K
$182.8K
How much do ai annotation jobs pay per year?
What is an AI Annotation job?
An AI Annotation job involves labeling, tagging, or annotating data, such as images, text, or audio, to train machine learning models. Annotators help improve AI accuracy by providing high-quality, structured data that algorithms use to learn patterns. Tasks may include identifying objects in images, transcribing speech, or classifying text-based content. This job is essential for developing AI applications like self-driving cars, chatbots, and image recognition systems.
What are the typical daily responsibilities of an AI Annotation specialist?
As an AI Annotation specialist, your typical day will involve accurately labeling, categorizing, or tagging large volumes of images, text, audio, or video data to train AI models according to project guidelines. You may work independently or as part of a team, using specialized annotation platforms and regularly reviewing your work to ensure quality and consistency. Collaboration with data scientists or project managers may be required to clarify ambiguous cases or update labeling criteria. You can expect periodic feedback and performance reviews to help refine your skills and ensure the data meets the project’s standards, making attention to detail and adaptability essential for success.
What are the key skills and qualifications needed to thrive in the Ai Annotation position, and why are they important?
To thrive as an AI Annotation professional, you need keen attention to detail, strong analytical skills, and a basic understanding of machine learning concepts, often supported by a high school diploma or relevant technical training. Familiarity with data labeling tools, annotation platforms such as Labelbox or Supervisely, and basic spreadsheet or database management is commonly required. Strong communication, time management, and the ability to maintain focus during repetitive tasks are standout soft skills. These abilities are crucial for producing high-quality, consistent data that supports the effective development and accuracy of AI models.

Full-time
Posted 15 days ago
Johns Hopkins Medicine rating
7.5
Based on 205 frontline employees who took The Breakroom Quiz
231st of 889 rated healthcare providers
Job description
PREP Research Associate
This position is part of the National Institute of Standards and Technology (NIST) Professional Research Experience Program (PREP). NIST recognizes that its research staff may want to collaborate with researchers at academic institutions on specific projects of mutual interest and, therefore, requires those institutions to be recipients of a PREP award. The PREP program involves staff from a wide range of backgrounds conducting scientific research across various fields. Individuals in this position will perform technical work supporting the collaboration's scientific research.
Research Title:
Reliability of Human and LLM Annotations for AI Risk Assessment
The work will entail:
This project focuses on using Large Language Models (LLMs) to provide annotations of evaluation data (a.k.a., LLM as judge), and the design of an Inter-Annotator Agreement study to assess the reliability of both human and LLM annotations. The candidate will explore assessing the indicators of a given AI-related risk, determining how to identify them, and providing annotators with examples to annotate the presence of various risks. The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the validity and reliability of AI-related risks in data annotation.
U.S. Citizen Preferred
Key responsibilities will include but are not limited to:
- Gain familiarity with existing literature on data annotation and LLM as judge
- Understand NIST's role and ongoing efforts in assessing and measuring the validity and reliability of AI-related risks in data annotation
- Contribute to developing an annotation framework for AI risk assessment
- Collaborate effectively with cross-functional and interdisciplinary stakeholders to ensure successful project outcomes
Deliverables
- Contributions to a NIST report that supports ongoing NIST AI evaluation efforts focused on the design of an Inter-Annotator Agreement to assess the reliability of both human and LLM annotations.
Qualifications
- Background in Computer Science, Data Science, or related field.
- Education level: Bachelor's or Graduate Degree
- Strong interest in data annotation and AI risks
- Familiarity with scientific reading and technical writing
Application Instructions
Please upload the following with your application:
• CV/Resume
*Please limit C.V to 3 pages only and ONLY include a valid email address for your contact info. Your resume will not be considered if the following information is included on your CV/resume.
• Self portraits
• Phone number
• Home address/Country
• Citizenship status
• Languages spoken
• Sex/Gender
Privacy Act Statement
Authority: 15 U.S.C. § 278g-1(e)(1) and (e)(3) and 15 U.S.C. § 272(b) and (c)
Purpose: The National Institute for Standards and Technology (NIST) hosts the Professional Research Experience Program (PREP) which is designed to provide valuable laboratory experience and financial assistance to undergraduates, post-bachelor's degree holders, graduate students, master's degree holders, postdocs, and faculty.
PREP is a 5-year cooperative agreement between NIST laboratories and participating PREP Universities to establish a collaborative research relationship between NIST and U.S. institutions of higher education in the following disciplines including (but may not be limited to) biochemistry, biological sciences, chemistry, computer science, engineering, electronics, materials science, mathematics, nanoscale science, neutron science, physical science, physics, and statistics. This collection of information is needed to facilitate administrative functions of the PREP Program.
Routine Uses: NIST will use the information collected to perform the requisite reviews of the applications to determine eligibility, and to meet programmatic requirements. Disclosure of this information is also subject to all the published routine uses as identified in the Privacy Act System of Records Notices: NIST-1: NIST Associates.
Disclosure: Furnishing this information is voluntary. When you submit the form, you are indicating your voluntary consent for NIST to use of the information you submit for the purpose stated.
What Johns Hopkins Medicine employees say
Pay
Benefits
Hours and flexibility
Workplace
Get the full story on Breakroom
About Johns Hopkins Hospital
Sourced by ZipRecruiter
Industry
Health care and social assistance
Company size
10,000+ Employees
Headquarters location
Baltimore, MD, US
Year founded
1889