ZipRecruiter

Log In

1

Llm Annotation Jobs in Washington (NOW HIRING)

Seekr

Senior Data Scientist

Reston, VA · On-site

... LLM fine-tuning). Additionally, this person will be working on quick proof-of-concept solutions but ... Coordinate data collection and annotation efforts for supervised training efforts * Design and ...

Omm IT Solutions

Senior Data Scientist

Ellicott City, MD · On-site

Train and optimize NLP\/LLM models and create Python based pipelines \n * Experience building cloud ... annotation tools and semantic frameworks. \n * Ability to clean and process large amounts of real ...

State Affairs

Software Engineer

Washington, DC · On-site

... annotation, and hiding terabytes behind sub-200 ms interactions. • Partner with the design and ... to utilize LLM tools (Claude-Code, Cursor, Codex) to accelerate coding and debugging Preferred ...

Axle

Data Scientist II

Rockville, MD

... annotation, DEG), and Digital Spatial Profiling (annotation, QC, normalization, spatial ... Incorporate AI/LLM-based extraction where appropriate, with clear validation and communication to ...

State Affairs

Software Engineer

Washington, DC · On-site

$120K - $210K/yr

Design delightful data workflow such as instant search, realtime diffing, collaborative annotation ... Ability to utilize LLM tools (Claude-Code, Cursor, Codex) to accelerate coding and debugging ...

next page

Showing results 1-20

All JobsLlm Annotation JobsLlm Annotation Jobs in Washington

Llm Annotation information

What is LLM annotation?

LLM annotation refers to the process of labeling or tagging data specifically for training and evaluating large language models (LLMs) like GPT or BERT. Annotators read text and apply labels, correct errors, or provide feedback to help improve the model's understanding and performance. This work is crucial for supervised learning, as well-annotated datasets help LLMs better recognize patterns, context, and meaning in human language. LLM annotation can involve tasks such as sentiment analysis, named entity recognition, or instruction following. Annotators often use specialized platforms or tools to complete their tasks efficiently and accurately.

What are the key skills and qualifications needed to thrive as an LLM annotation specialist?

To thrive as an LLM Annotation Specialist, you need strong analytical skills, attention to detail, and a background in linguistics, computer science, or a related field. Familiarity with annotation platforms, natural language processing (NLP) tools, and data labeling systems is typically required. Excellent communication, critical thinking, and the ability to follow guidelines precisely are valuable soft skills for this role. These skills ensure high-quality, accurate data annotation, which directly impacts the performance and reliability of large language models.

What are some common challenges faced by LLM annotation specialists, and how can they be addressed?

LLM Annotation specialists often encounter challenges such as interpreting ambiguous language data, maintaining annotation consistency across complex datasets, and keeping up with evolving guidelines. These can be addressed by participating in regular team syncs to clarify guidelines, using annotation tools with built-in quality checks, and collaborating closely with project leads and fellow annotators. Continuous learning and open communication help ensure high-quality, reliable data annotation and support professional growth within the AI and NLP fields.

What is the difference between Llm Annotation vs Data Labeler?

AspectLlm AnnotationData Labeler
Required CredentialsBasic computer skills, sometimes familiarity with AI toolsBasic skills, often on-the-job training
Work EnvironmentRemote or office-based, tech-focusedRemote or on-site, varied industries
Industry UsageAI, machine learning, NLP projectsVarious industries including marketing, healthcare, and tech
Search & Comparison IntentUnderstanding roles in AI data preparationGeneral data labeling tasks

In summary, Llm Annotation involves specialized annotation for large language models, often requiring familiarity with AI tools, while Data Labeler is a broader role focused on labeling data across multiple industries with minimal technical requirements.

How to become an Llm annotator?

To become an LLM annotator, candidates typically need strong language skills, attention to detail, and familiarity with data annotation tools. Many positions require a high school diploma or equivalent, and some companies provide training. Experience with machine learning or natural language processing can be beneficial but is not always necessary.

What are popular job titles related to Llm Annotation jobs in Washington?

For Llm Annotation jobs in Washington, the most frequently searched job titles are:

What job categories do people searching Llm Annotation jobs in Washington look for?

The top searched job categories for Llm Annotation jobs in Washington are:

What cities in Washington are hiring for Llm Annotation jobs?

Cities in Washington with the most Llm Annotation job openings:

Llm Annotation jobs near you
Infographic showing various Llm Annotation job openings in Washington as of August 2026, with employment types broken down into 93% Full Time, 3% Part Time, and 4% Contract. Highlights an 79% Physical, 5% Hybrid, and 16% Remote job distribution.

Assistant Research Scientist (PREP0004176)

Gaithersburg, MD • On-site

Johns Hopkins University
Colleges, Universities, and Professional Schools • 10K+ employees

8.0

Company rating: 8.0 out of 10

Based on 71 frontline employees who took The Breakroom Quiz

192nd of 625 rated colleges and universities

People enjoy working here

Good employer

Recommended by students

Learn more about Johns Hopkins Universityexternal

Full-time

Re-posted 22 days ago

Job description

Description
PREP Research Associate
This position is part of the National Institute of Standards and Technology (NIST) Professional Research Experience Program (PREP). NIST recognizes that its research staff may want to collaborate with researchers at academic institutions on specific projects of mutual interest and, therefore, requires those institutions to be recipients of a PREP award. The PREP program involves staff from a wide range of backgrounds conducting scientific research across various fields. Individuals in this position will perform technical work supporting the collaboration's scientific research.
Research Title:
Reliability of Human and LLM Annotations for AI Risk Assessment
The work will entail:
This project focuses on using Large Language Models (LLMs) to provide annotations of evaluation data (a.k.a., LLM as judge), and the design of an Inter-Annotator Agreement study to assess the reliability of both human and LLM annotations. The candidate will explore assessing the indicators of a given AI-related risk, determining how to identify them, and providing annotators with examples to annotate the presence of various risks. The project aims to develop an annotation framework for AI risk assessment and establish metrics for data quality in AI risk research, supporting broader work at NIST in assessing and measuring the validity and reliability of AI-related risks in data annotation.
U.S. Citizen Preferred
Key responsibilities will include but are not limited to:
  • Gain familiarity with existing literature on data annotation and LLM as judge
  • Understand NIST's role and ongoing efforts in assessing and measuring the validity and reliability of AI-related risks in data annotation
  • Contribute to developing an annotation framework for AI risk assessment
  • Collaborate effectively with cross-functional and interdisciplinary stakeholders to ensure successful project outcomes

Deliverables
  • Contributions to a NIST report that supports ongoing NIST AI evaluation efforts focused on the design of an Inter-Annotator Agreement to assess the reliability of both human and LLM annotations.

Qualifications
  • Background in Computer Science, Data Science, or related field.
  • Education level: Bachelor's or Graduate Degree
  • Strong interest in data annotation and AI risks
  • Familiarity with scientific reading and technical writing

Application Instructions
Please upload the following with your application:
• CV/Resume
*Please limit C.V to 3 pages only and ONLY include a valid email address for your contact info. Your resume will not be considered if the following information is included on your CV/resume.
• Self portraits
• Phone number
• Home address/Country
• Citizenship status
• Languages spoken
• Sex/Gender
Privacy Act Statement
Authority: 15 U.S.C. § 278g-1(e)(1) and (e)(3) and 15 U.S.C. § 272(b) and (c)
Purpose: The National Institute for Standards and Technology (NIST) hosts the Professional Research Experience Program (PREP) which is designed to provide valuable laboratory experience and financial assistance to undergraduates, post-bachelor's degree holders, graduate students, master's degree holders, postdocs, and faculty.
PREP is a 5-year cooperative agreement between NIST laboratories and participating PREP Universities to establish a collaborative research relationship between NIST and U.S. institutions of higher education in the following disciplines including (but may not be limited to) biochemistry, biological sciences, chemistry, computer science, engineering, electronics, materials science, mathematics, nanoscale science, neutron science, physical science, physics, and statistics. This collection of information is needed to facilitate administrative functions of the PREP Program.
Routine Uses: NIST will use the information collected to perform the requisite reviews of the applications to determine eligibility, and to meet programmatic requirements. Disclosure of this information is also subject to all the published routine uses as identified in the Privacy Act System of Records Notices: NIST-1: NIST Associates.
Disclosure: Furnishing this information is voluntary. When you submit the form, you are indicating your voluntary consent for NIST to use of the information you submit for the purpose stated.
Johns Hopkins University logo

About Johns Hopkins University

Sourced by ZipRecruiter

Gilman believed that teaching and research go hand in hand—that success in one depends on success in the other—and that a modern university must do both well. He also believed that sharing our knowledge and discoveries would help make the world a better place. In 145 years, we haven’t strayed from that vision. This is still a destination for excellent, ambitious scholars and a world leader in teaching and research. Distinguished professors mentor students in the arts and music, humanities, social and natural sciences, engineering, international studies, education, business, and the health professions. Those same faculty members, along with their colleagues at the university’s Applied Physics Laboratory, have made us the nation’s leader in federal research and development funding every year since 1979. That’s a fitting distinction for America’s first research university, a place that has revolutionized higher education in the U.S. and continues to bring knowledge and discoveries to the world.

Industry

Colleges, universities, and professional schools

Company size

10,000+ Employees

Headquarters location

Baltimore, MD, US

Year founded

1876

View All Johns Hopkins University Jobs

What Johns Hopkins University employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom

Apply