ZipRecruiter

Log In

1

Ai Rater Jobs in New York (NOW HIRING)

Thomas Talent Network

AI Agent Engineer

New York, NY

$200K - $300K/yr

With a team of five, it scaled to a $2M annualized run rate in just a few months, targeting $10M ARR this year. About the Role Building its voice AI agent infrastructure in-house and needs an AI ...

SuperDial

AI Deployment Lead

New York, NY · On-site

Track and improve key trial metrics, including batch completion rate, trial success rate ... Use AI tools, automation, and structured workflows to reduce manual effort, improve consistency ...

Brex

Manager, CX AI Strategy

New York, NY · On-site

Brex's AI-native automation and world-class service eliminate manual expense and accounting tasks ... Collaborate with CX leaders to drive performance metrics: resolution rate, CSAT, quality ...

Cantor Fitzgerald

Senior AI Research Engineer

Manhattan, NY · On-site

$115K - $158K/yr

Design, build, evaluate, and maintain production-grade AI applications for FMX's rates and derivatives exchange business. * Develop the Skynapse orchestration layer, including task decomposition ...

New

Humana

Senior Applied AI Software Engineer ( AI)

New York, NY · On-site +1

$134K - $176K/yr

Experience with AI system evaluation, observability, reliability, fallback strategies, safety guardrails, latency optimization, and rate-limit management within enterprise production environments.

Humana

Senior Applied AI Software Engineer ( AI)

New York, NY · On-site +1

$134K - $176K/yr

Experience with AI system evaluation, observability, reliability, fallback strategies, safety guardrails, latency optimization, and rate-limit management within enterprise production environments.

Humana

Senior Applied AI Software Engineer ( AI)

New York, NY · On-site +1

$134K - $176K/yr

Experience with AI system evaluation, observability, reliability, fallback strategies, safety guardrails, latency optimization, and rate-limit management within enterprise production environments.

Rebel Hotel Company

AI Engineer

Manhattan, NY · Remote

Today our commercial teams run rate strategy, RFP negotiations, and account planning on spreadsheets and intuition. We're hiring an AI Engineer to turn that work into software -- models and tools ...

Rebel Hospitality New York LLC

AI Engineer

New York, NY · On-site

$190K - $200K/yr

Today our commercial teams run rate strategy, RFP negotiations, and account planning on spreadsheets and intuition. We're hiring an AI Engineer to turn that work into software - models and tools that ...

David Joseph & Company

AI Agent Engineer

New York, NY

$200K - $300K/yr

Its AI agents handle the full late-stage recovery process -- asset research, pre-legal outreach ... Fast traction: ~$2M annualized run rate in a few months with a team of five, targeting $10M ARR ...

previous page

Showing results 41-60

All JobsAi Rater JobsAi Rater Jobs in New York

Ai Rater information

What is an AI Rater?

An AI Rater evaluates and provides feedback on artificial intelligence models, typically improving search engines, chatbots, or recommendation systems. They assess the relevance, accuracy, and quality of AI-generated content based on specific guidelines. This role requires strong analytical skills, attention to detail, and familiarity with the subject matter being reviewed. AI Raters often work remotely and on a flexible schedule.

What does an AI Rater do?

A typical day for an AI Rater involves reviewing and evaluating various types of content, such as search engine results, social media posts, advertisements, or chatbot responses, to ensure they meet quality and relevancy standards. You may follow detailed guidelines to rate or annotate content, complete assigned tasks in a web-based platform, and provide feedback to help improve AI performance. Most positions are remote and offer flexible schedules, allowing you to plan your workload around personal commitments. Collaboration is generally limited, as most work is performed independently, but periodic communication with team leads for training or updates is common.

What skills and qualifications are needed to thrive as an AI Rater?

To thrive as an AI Rater, you generally need strong attention to detail, analytical thinking, and proficiency in English, often supported by formal education such as a high school diploma or higher. Familiarity with web browsers, online research, and company-specific rating platforms or guidelines is essential. Excellent time management, adaptability, and effective written communication help individuals excel in this position. These skills and qualities ensure accurate and consistent evaluations of AI-generated content, directly impacting the improvement of artificial intelligence systems.

How to become an AI Rater?

To become an AI Rater, candidates typically need a high school diploma or equivalent, strong language and analytical skills, and the ability to follow detailed guidelines. The role often involves evaluating search engine results or content for accuracy and relevance, using online platforms or specific assessment tools. Prior experience in data annotation or quality assurance can be beneficial, and flexible scheduling is common.

What are the most commonly searched types of Ai Rater jobs in New York?

The most popular types of Ai Rater jobs in New York are:

What are popular job titles related to Ai Rater jobs in New York?

For Ai Rater jobs in New York, the most frequently searched job titles are:

What job categories do people searching Ai Rater jobs in New York look for?

The top searched job categories for Ai Rater jobs in New York are:

What cities in New York are hiring for Ai Rater jobs?

Cities in New York with the most Ai Rater job openings:

Ai Rater jobs near you
Infographic showing various Ai Rater job openings in New York as of August 2026, with employment types broken down into 47% Full Time, and 53% Part Time. Highlights an 100% Remote job distribution.

AI Agent Engineer

Thomas Talent Network

New York, NY

$200K - $300K/yr

Full-time

Posted 9 days ago

Job description

About the Company

A fast-growing, YC-backed seed-stage startup (founded 2024) using generative AI to automate debt collection — a $35B US market. Its AI agents already outperform human collectors 2x at a fraction of the cost, doing 10M+ calls per month. With a team of five, it scaled to a $2M annualized run rate in just a few months, targeting $10M ARR this year.

About the Role

Building its voice AI agent infrastructure in-house and needs an AI Agent Engineer to own it end-to-end. You'll build on top of LiveKit, design the orchestration layer, own the prompting strategy for millions of consumer interactions, and continuously improve voice-agent performance through rigorous testing and iteration. A highly technical role for someone equally comfortable building production-grade systems and writing the conversational AI logic that powers real-world outcomes. Deep voice-AI experience is ideal; a strong infrastructure engineer who is analytical and business-minded is also a fit.

What You'll Own
  • Build the voice AI infrastructure in LiveKit from the ground up
  • Design and own the architecture of real-time voice-agent systems: orchestration, state management, latency optimization, prompting, reliability
  • Own the end-to-end prompting strategy: conversation flows, tone, compliance-safe phrasing, guardrails, edge-case handling
  • Build and maintain testing/evaluation frameworks (A/B testing, regression testing, conversation QA, benchmarking, KPI reporting)
  • Write, expand, and continuously improve prompting to cover more real-world scenarios
  • Iterate on performance data and call outcomes to improve containment, conversion, liquidation rates, and satisfaction
  • Optimize ASR, STT, TTS, turn-taking, interruption handling, and state transitions
  • Build scalable systems/tooling for millions of voice interactions; migrate clients off legacy voice infra while preserving customizations
  • Work with product, operations, and leadership to translate business goals into high-performing agent behavior
Must-Have
  • 2+ years in voice AI, conversational AI, prompt engineering, speech systems, or related agentic workflows
  • 3+ years in platform, infrastructure, or backend engineering
  • Strong experience with Node.js, AWS, and SQL
  • Experience building and shipping production systems in a startup or similarly fast-paced environment
  • Experience with voice orchestration platforms or agent frameworks such as LiveKit or equivalent
  • Strong prompt-writing instincts; designs clear, effective, resilient conversational flows
  • Strong analytical skills; ties technical work directly to business outcomes and ROI
  • Excellent written and verbal English (required for the prompt-engineering component)
  • High ownership and strong work ethic
  • Based in or willing to relocate to NYC; in-person 4–5 days per week
Nice-to-Have
  • SIP trunking, telephony infrastructure, and production voice systems
  • Real-time audio pipelines, call routing, carrier/telephony integrations
  • Speech infrastructure in production environments
  • Domain knowledge in debt collection, financial services, or regulated consumer communication
Details

Employment: Full-Time | Location: Manhattan, NY — In-person (4–5 days/week) | Experience: 3–10 years
Compensation: $200,000–$300,000 / year + equity. No visa sponsorship.