ZipRecruiter

Log In

1

Permanent Ai Annotation Writing Jobs in Washington

Innodata Inc.

QA / Evaluation Lead

Washington, DC

$45 - $50/hr

Support AI Solutions Engineer on evaluation design for SAM 2 and Frontier model API validation ... CVAT or equivalent annotation platform QC workflow configuration * Drift detection and model ...

previous pagenext page

Showing results 21-40

All JobsPermanent Ai Annotation Writing JobsPermanent Ai Annotation Writing Jobs in Washington

Permanent Ai Annotation Writing information

What is permanent AI annotation writing?

Permanent AI Annotation Writing jobs involve the ongoing task of labeling and annotating data—such as text, images, audio, or video—so that artificial intelligence systems can learn and improve. Annotators use specialized tools to identify objects, actions, or features in datasets, providing crucial information that helps train AI models. These positions are typically long-term or full-time, offering stability and the opportunity to develop expertise in AI data preparation. Workers in this field often collaborate with data scientists and engineers to ensure high-quality, accurate annotations. The demand for skilled annotators is growing as AI applications expand into more industries.

What are some typical challenges faced by professionals in permanent AI annotation writing roles, and how can they be addressed?

In permanent AI annotation writing roles, professionals often encounter challenges such as maintaining high accuracy and consistency across large volumes of data, adapting to evolving project guidelines, and managing repetitive tasks. To address these challenges, it's important to develop a systematic approach to annotation, regularly review updated instructions, and collaborate closely with quality assurance teams. Utilizing productivity tools and participating in team discussions can also help streamline workflows and reduce errors, ensuring that the annotated data meets the required standards for AI model training.

What are the key skills and qualifications needed to thrive as a permanent AI annotation writer?

To thrive as a Permanent AI Annotation Writer, you need excellent attention to detail, strong language proficiency, and a basic understanding of data labeling principles, often supported by relevant coursework or experience in linguistics or data science. Familiarity with annotation tools like Labelbox or Prodigy, and experience following annotation guidelines or taxonomies, are commonly required. Strong communication, critical thinking, and the ability to work independently are valuable soft skills in this role. These skills ensure high-quality, consistent data labeling, which directly impacts the performance and reliability of AI models.
What are the most commonly searched types of Ai Annotation Writing jobs in Washington? The most popular types of Ai Annotation Writing jobs in Washington are:
What are popular job titles related to Permanent Ai Annotation Writing jobs in Washington? For Permanent Ai Annotation Writing jobs in Washington, the most frequently searched job titles are:
What job categories do people searching Permanent Ai Annotation Writing jobs in Washington look for? The top searched job categories for Permanent Ai Annotation Writing jobs in Washington are:
What cities in Washington are hiring for Permanent Ai Annotation Writing jobs? Cities in Washington with the most Permanent Ai Annotation Writing job openings:
Permanent Ai Annotation Writing jobs near you

QA / Evaluation Lead

Innodata Inc.

Washington, DC

$45 - $50/hr

Other

Posted 27 days ago

Innodata rating

7.5

Company rating: 7.5 out of 10

Based on 6 frontline employees who took The Breakroom Quiz

161st of 242 rated software companies

Job description

About the Program: 

Innodata's Federal Practice builds the trusted data layer for critical infrastructure Trust & Safety work. Partnering with a leading systems integrator, we're delivering a modern, governed data services platform in a secure federal (IL4) environment. Over an intensive 20-week phase, you'll help stand up a data services storefront, a DataCard governance framework, synthetic data integration, and Databricks write-back capabilities.

About the Role: 

As the QA/Evaluation Lead, you'll own quality and evaluation across the platform. You'll design the evaluation framework that measures whether our data services and outputs meet the bar, build repeatable test and validation processes, and give the team an objective read on readiness at each milestone. Partnering with the Delivery Owner and engineering leads, you'll turn quality from an afterthought into a measurable, demonstrable strength. It's a role for someone who thinks rigorously about evaluation and takes pride in evidence-backed quality.

Key Responsibilities:

  • Design and own the inter-annotator agreement (IAA) methodology for the Phase 1 demonstration corpus - metric selection (Cohen's kappa, Fleiss, Krippendorff's alpha), sampling design, adjudication workflow, and agreement thresholds
  • Define evaluation framework architecture: test and evaluation plans, IAA targets, drift detection gates, and model performance metrics per SOW Section 2.9
  • Configure and operate sampling-based quality control across the self-service and white-glove annotation paths during Phase D corpus production
  • Design and implement confidence-threshold escalation routing from automated annotation to senior-annotator adjudication
  • Validate quality scoring and IAA computation within the Innodata data layer
  • Support AI Solutions Engineer on evaluation design for SAM 2 and Frontier model API validation - define what 'good enough' looks like quantitatively
  • Produce evaluation framework documentation for the Phase 1 NPP closeout package, including per-DataCard documentation with the SA

Must-Have Qualifications:

  • Bachelor's degree in Statistics, Data Science, Computer Science, or related quantitative field required; Master's degree preferred. Equivalent experience may substitute for degree on a 2-for-1 basis.
  • 6+ years total professional experience, 4+ years in data quality, evaluation methodology, or QA on AI/ML programs
  • IAA methodology expertise - Cohen's kappa, Fleiss' kappa, Krippendorff's alpha: hands-on, not theoretical
  • Evaluation framework design for AI/ML training data programs
  • QC process design: sampling methodology, escalation workflows, adjudication protocols
  • Python for QC tooling, metric computation, and statistical analysis
  • Active Secret clearance with TS/SCI eligibility

Nice-to-Have Qualifications:

  • Prior DoD or IC data quality program experience
  • CVAT or equivalent annotation platform QC workflow configuration
  • Drift detection and model monitoring methodology
  • Experience with FMV / video annotation quality standards

The expected hourly salary range for this position is $45 to $50 p/hour, based on experience, skills, and qualifications.

Note to Candidates: 

This role is not a project manager with QC responsibilities - it is a methodology expert who owns the intellectual framework behind data quality on a federal AI program. The right candidate can walk into a meeting with Government evaluators and explain exactly why the evaluation design produces trustworthy labels. That conversation is part of Phase 2 positioning.

What Innodata employees say

Hours and flexibility

Workplace

Get the full story on Breakroom

Apply