ZipRecruiter

Log In

1

Ai Evaluator Jobs (NOW HIRING)

Mercor

AI Evaluator - Software Expert

New York, NY · Remote

$80 - $120/hr

Software / AI / IT / data Evaluator Type: Contract Compensation: $80-$120/hour Location: Remote Role Responsibilities * Evaluate AI-generated artifacts against domain-specific quality rubrics.

New

Mercor

Financial Compliance AI Evaluator

New York, NY · Remote

$80 - $120/hr

Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80-$120/hour Location: Remote Role Responsibilities * Evaluate AI-generated artifacts against ...

Mercor

Compliance AI Evaluator - Finance

New York, NY · Remote

$80 - $120/hr

Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80-$120/hour Location: Remote Role Responsibilities * Evaluate AI-generated artifacts against ...

next page

Showing results 1-20

All JobsAi Evaluator Jobs

Ai Evaluator information

See salary details

$29.5K

$65.5K

$106.5K

How much do ai evaluator jobs pay per year?

As of Aug 14, 2026, the average yearly pay for ai evaluator in the United States is $65,471.00, according to ZipRecruiter salary data. Most workers in this role earn between $44,500.00 and $79,500.00 per year, depending on experience, location, and employer.

How much do AI evaluators make?

AI evaluators typically earn between $40,000 and $80,000 annually, depending on experience, location, and the complexity of tasks. Many roles are freelance or part-time, with some positions offering flexible schedules and requiring skills in data analysis and machine learning tools.

What are the key skills and qualifications needed to thrive as an AI evaluator?

To thrive as an AI Evaluator, you need a solid understanding of machine learning concepts, data analysis, and critical thinking, often supported by a degree in computer science, data science, or a related field. Familiarity with annotation tools, model assessment platforms, and sometimes programming languages like Python is highly advantageous. Attention to detail, strong communication skills, and the ability to provide constructive feedback are essential soft skills. These competencies are crucial for accurately assessing AI model outputs and collaborating with development teams to improve system performance.

How to become an AI evaluator?

To become an AI evaluator, candidates typically need a background in computer science, data analysis, or related fields, along with strong analytical skills. Experience with machine learning, natural language processing, or data annotation tools is often required, and some roles may require certification or training in AI or data labeling platforms.

What is an AI evaluator?

An AI Evaluator assesses the performance, accuracy, and relevance of artificial intelligence models, often focusing on language models, search engines, or recommendation systems. They review AI-generated responses, ensuring they align with human expectations and industry standards. Evaluators may follow specific guidelines to rate or provide feedback on outputs, helping improve AI functionality. This role typically requires analytical skills, attention to detail, and proficiency in the target language. Many AI Evaluators work remotely on a contract or freelance basis.

What are some typical challenges an AI evaluator faces in their daily work?

AI Evaluators often encounter complex data or ambiguous responses that require nuanced judgment to assess accurately. Adapting to rapidly evolving guidelines and learning new toolsets as AI projects progress is also common. Routine tasks may include evaluating AI-generated outputs for relevance and bias, documenting errors, and discussing findings with engineers or linguists. While the work is intellectually stimulating, it demands precision and flexibility to ensure models meet high-quality standards.

More about Ai Evaluator jobs

What cities are hiring for Ai Evaluator jobs?

Cities with the most Ai Evaluator job openings:

What are the most commonly searched types of Ai Evaluator jobs?

The most popular types of Ai Evaluator jobs are:

What states have the most Ai Evaluator jobs?

States with the most job openings for Ai Evaluator jobs include:

What job categories do people searching Ai Evaluator jobs look for?

The top searched job categories for Ai Evaluator jobs are:

Ai Evaluator jobs near you
Infographic showing various Ai Evaluator job openings in the United States as of August 2026, with employment types broken down into 33% Internship, 34% Full Time, and 33% Part Time. Highlights an 33% In-person, and 67% Remote job distribution, with an average salary of $65,471 per year, or $31.5 per hour.

AI Evaluator - Software Expert

Mercor

New York, NY • Remote

$80 - $120/hr

Full-time

Posted 2 days ago

New

Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated artifacts against domain-specific quality rubrics.
  • Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks.
  • Provide clear, structured written feedback to improve AI model outputs.
  • Apply deep subject-matter expertise to grade outputs for accuracy and rigor.
  • Work independently and asynchronously to meet deadlines.

Qualifications

Must-Have

  • 5+ years of relevant professional experience in Software / AI / IT / data.
  • Native or professional fluency in English.
  • Highly proficient in Microsoft Office and Google Workspace, especially Slides.

Preferred

  • Advanced degree (Master's or higher) from a reputable institution.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.