1

Ai Rater Jobs in California (NOW HIRING)

Design and implement REST/GraphQL APIs with proper auth (OAuth2/JWT), input validation, and rate ... AI-native building ability: you know how to use AI tools (Claude Code, Cursor, Copilot, etc.) to ...

Nexxa is building the best AI systems for heavy industries -- enabling machines, systems and ... Define and track quality metrics beyond simple accuracy -- groundedness, hallucination rate, task ...

Nexxa is building the best AI systems for heavy industries - enabling machines, systems and ... Define and track quality metrics beyond simple accuracy - groundedness, hallucination rate, task ...

QA Engineer (AI Systems)

Sunnyvale, CA · On-site

$150 - $190/hr

Nexxa is building the best AI systems for heavy industries -- enabling machines, systems and ... Define and track quality metrics beyond simple accuracy -- groundedness, hallucination rate, task ...

Integrate AI models developed by AI engineers into product backends while continuously improving scalability, monitoring, and resilience based on latency, error rates, and operational costs. * Domain ...

Head of AI

San Francisco, CA · On-site

$250K - $350K/yr

Own output quality across Highlight chat, actions, and workflows; drive toward industry leading acceptance rates. * Build and lead a world-class ML Engineering team; Establish AI/ML best practices ...

next page

Showing results 1-20

Ai Rater information

What is an AI Rater?

An AI Rater evaluates and provides feedback on artificial intelligence models, typically improving search engines, chatbots, or recommendation systems. They assess the relevance, accuracy, and quality of AI-generated content based on specific guidelines. This role requires strong analytical skills, attention to detail, and familiarity with the subject matter being reviewed. AI Raters often work remotely and on a flexible schedule.

What does an AI Rater do?

A typical day for an AI Rater involves reviewing and evaluating various types of content, such as search engine results, social media posts, advertisements, or chatbot responses, to ensure they meet quality and relevancy standards. You may follow detailed guidelines to rate or annotate content, complete assigned tasks in a web-based platform, and provide feedback to help improve AI performance. Most positions are remote and offer flexible schedules, allowing you to plan your workload around personal commitments. Collaboration is generally limited, as most work is performed independently, but periodic communication with team leads for training or updates is common.

What skills and qualifications are needed to thrive as an AI Rater?

To thrive as an AI Rater, you generally need strong attention to detail, analytical thinking, and proficiency in English, often supported by formal education such as a high school diploma or higher. Familiarity with web browsers, online research, and company-specific rating platforms or guidelines is essential. Excellent time management, adaptability, and effective written communication help individuals excel in this position. These skills and qualities ensure accurate and consistent evaluations of AI-generated content, directly impacting the improvement of artificial intelligence systems.

How to become an AI Rater?

To become an AI Rater, candidates typically need a high school diploma or equivalent, strong language and analytical skills, and the ability to follow detailed guidelines. The role often involves evaluating search engine results or content for accuracy and relevance, using online platforms or specific assessment tools. Prior experience in data annotation or quality assurance can be beneficial, and flexible scheduling is common.

What are the most commonly searched types of Ai Rater jobs in California?

The most popular types of Ai Rater jobs in California are:

What are popular job titles related to Ai Rater jobs in California?

For Ai Rater jobs in California, the most frequently searched job titles are:

What job categories do people searching Ai Rater jobs in California look for?

The top searched job categories for Ai Rater jobs in California are:

What cities in California are hiring for Ai Rater jobs?

Cities in California with the most Ai Rater job openings:

Infographic showing various Ai Rater job openings in California as of August 2026, with employment types broken down into 77% Full Time, 20% Part Time, and 3% Contract. Highlights an 66% Physical, 4% Hybrid, and 30% Remote job distribution.

AI Engineer

O2 AI

Mountain View, CA • On-site

Other

Posted 5 days ago


Key responsibilities

  • Participate in end-to-end product and engineering workflows, including building and improving agentic systems and evaluation pipelines.

  • Design, implement, and ship backend code such as APIs, database schemas, and permission models for production systems.

  • Operate and maintain deployed services by containerizing, deploying, and setting up monitoring and logging.


Job description

At O₂ AI, we are building a Vertical AI Agent for the global semiconductor and electronics supply chain, just as Harvey is transforming the legal industry.

Company Website: https://o2tech.ai/

We build deeply vertical AI Agents capable of handling high-stakes, high-complexity challenges in global manufacturing systems, transforming chaos into efficient, intelligent automated execution.

We are looking for a hands-on Builder to own product engineering end-to-end: from understanding real customer problems, to designing and shipping AI-native systems, to iterating in production. 


Why Join Us?

Elite Founding Team

Work alongside founders & early members from Yale, Northwestern, NUS, Columbia, Stanford and MIT, with experience across DeepSeek, NVIDIA, and Microsoft, etc.. We are a team that cares equally about frontier AI and real product execution.

We ship: (1) systems used by real customers, (2) awarded publications at top venues. 

Real Customers, Real Impact

At O₂ AI, we provide you with direct opportunities to communicate with customers. You will interface directly with US clients and collaborate with Fortune 500 electronics and semiconductor companies. Our products span three layers:

  • Electronic Vertical AI Copilot — domain-specific knowledge assistance that understands industry context deeply enough to be trusted by experts.
  • ERP Workflow Agent — automating complex operational workflows with real permissions, real data flows, and real state management.
  • Pricing Decision Agent — high-stakes decision support where accuracy and reliability directly affect business outcomes.

Your code will touch real users, real operational constraints, and real revenue.

True 0 → 1 Ownership

At O₂ AI, we create opportunities for you to deeply engage in product and engineering. You will participate in the complete process from problem definition and solution design to product launch and continuous iteration, gaining highly hands-on practical experience and witnessing the real-world impact of your work in customer business scenarios.

Co-creating Cutting-edge AI with Frontier Labs

O₂ Lab, our downstream research-driven lab, collaborates deeply with Northwestern MLL Lab and MIT Media Lab. Members of the founding team have been recognized at venues such as NeurIPS LAW (Outstanding Paper) and ICCV SP4V (Best Paper), with 10k+ github stars, Huggingface #2 paper of the day, and invited talks at DeepMind, MIT, etc.

We focus on translating research into production-grade vertical AI systems. You will have opportunities for joint research, publications, and turning research insights into shipped products. 


What You'll Work On

Participate in end-to-end product and engineering workflows. You are expected to participate in at least one of the following terms in the product lifecycle:

Build and improve agentic workflows and systems

  • Agent architectures: context engineering, harness engineering (orchestration logic, routing, fallback chains, state management across turns).
  • Backend Services: implement tool/function calling integrations (APIs, databases, internal services) and handle the messy edge cases: timeouts, malformed responses, cost tracking.
  • Evaluation pipelines: think about failure modes like prompt injection, data leakage through context, runaway agent loops, cost blowups.
  • Research-driven exploration: LLM agent training, reinforcement learning, continual learning for evolving domains, improving performance and publishing papers / open-source projects.

Ship backend code in a real production system

  • Design and implement REST/GraphQL APIs with proper auth (OAuth2/JWT), input validation, and rate limiting. 
  • Write and migrate SQL schemas, normalize where it matters, denormalize where performance demands it. 
  • Implement RBAC or ABAC permission models that actually map to business requirements.

Operate what you build

  • Containerize services (Docker), write CI/CD pipelines, deploy to AWS/GCP. 
  • Set up logging, metrics, and alerts.


Foundation We Hope You Have

Strong coding fundamentals: at minimum, solid in Python. You can read, write, debug, and reason about code under pressure, with strong experiences coding agent workflow, training, or evaluation.

AI-native building ability: you know how to use AI tools (Claude Code, Cursor, Copilot, etc.) to prototype, iterate, and ship quickly. You don't just prompt, but build systems around prompts.

Fast learner: you can pick up new stacks, APIs, codebases, and business domains with minimal hand-holding. 

Product and systems sense: you can look at a messy business process and start to see where the data flows, where the agent should act, and where a human should stay in the loop.

Ownership under ambiguity: Ask the right questions, not waiting until everything is ready, make reasonable assumptions, build a prototype, and iterate from there.

Nice to have:

General Computer Science background (CS / EE / Software Engineering / AI, etc.)

Familiar with relevant technical stacks (React + TypeScript ; Python / Node.js)

Strong background in AI / LLM / Agent systems (hackathon awards, course project awards, strong GPA in CS/AI/ML courses, publications, open-source projects, etc)

Familiar with or willing to dive deep into:

  • Context Engineering / Harness Engineering
  • Multi-step reasoning and Agent behavior
  • General Problem solving

Patience and enthusiasm for writing code, debugging systems, and solving real problems


We Welcome High-Potential Builders!

If you don't yet have extensive experience or haven't built a complete production-grade product from 0 to 1, that's totally fine.

At O₂ AI, we are very happy to work with engineers who have solid coding abilities and learn quickly to build products together. This is an environment that allows you to truly participate in real systems and grow rapidly.

As long as you have a solid computer science foundation, good engineering discipline, and a strong desire to build, this will be an exceptionally valuable hands-on opportunity!

Work Location

Remote / Mountain View, California

Resume submission: engineer@o2tech.ai

If you have GitHub / portfolio / links to deployed systems, please feel free to include them!