1

Ai Rater Jobs in California (NOW HIRING)

Help Agent resolution rate and quality scores (completeness, correctness, relevance) improve ... AI & Tooling functions as a center of excellence for generative AI evaluation and tooling practices ...

AI Support Lead

San Francisco, CA · On-site

$200K - $280K/yr

Fluent in the metrics that prove a program is working: containment rate, deflection rate, resolution quality, CSAT on AI-assisted interactions, human escalation rate - and comfortable building and ...

Responsibilities : • Design, develop, and deploy AI agents and multi-agent systems to automate ... Acra Lending specializes in lending services and adjustable-rate mortgages as well as alternative ...

Agentic AI Engineer

Santa Clara, CA · On-site

$119K - $157K/yr

Please share your best matching candidates along with their resume, rate, earliest DOJ, and interview availability at the earliest. Title: Lead Agentic AI Engineer Location: Santa Clara, CA Job Type:

New

AI ARCHITECT (Redwood City, CA) *** We do not sponsor employment/work visas at this time, if you ... Rate discount on home and auto loans * Opportunity to use company owned condo in Maui and Lake ...

Define and track key performance indicators for each deployed AI tool, time saved, error rate, adoption rate, cost per transaction and report findings to CTO on a regular cadence. * Monitor AI ...

AI Architect

Redwood City, CA · On-site

$108K - $123K/yr

AI ARCHITECT (Redwood City, CA) *** We do not sponsor employment/work visas at this time, if you ... Rate discount on home and auto loans * Opportunity to use company owned condo in Maui and Lake ...

Define and track key performance indicators for each deployed AI tool, time saved, error rate, adoption rate, cost per transaction and report findings to CTO on a regular cadence. * Monitor AI ...

Talent AI Operator

Los Angeles, CA · On-site

$20.25 - $24.50/hr

As a Talent AI Operator , you are not a traditional recruiter. You are the human layer that makes ... Audit candidate matches, outreach quality, and conversion rates across thousands of weekly ...

Collaborate with CX leaders to drive performance metrics: resolution rate, CSAT, quality ... Own the CX AI roadmap from opportunity discovery through implementation - prioritize where AI can ...

AI Engineer

Aliso Viejo, CA · On-site

$65 - $70/hr

Demonstrated experience building AI agents with LangGraph. * Familiarity with at least one ... Benefits eligibility, accrual rates, and usage limits may vary based on employment status, length ...

Brex's AI-native automation and world-class service eliminate manual expense and accounting tasks ... Collaborate with CX leaders to drive performance metrics: resolution rate, CSAT, quality ...

Showing results 21-40

Ai Rater information

What is an AI Rater?

An AI Rater evaluates and provides feedback on artificial intelligence models, typically improving search engines, chatbots, or recommendation systems. They assess the relevance, accuracy, and quality of AI-generated content based on specific guidelines. This role requires strong analytical skills, attention to detail, and familiarity with the subject matter being reviewed. AI Raters often work remotely and on a flexible schedule.

What skills and qualifications are needed to thrive as an AI Rater?

To thrive as an AI Rater, you generally need strong attention to detail, analytical thinking, and proficiency in English, often supported by formal education such as a high school diploma or higher. Familiarity with web browsers, online research, and company-specific rating platforms or guidelines is essential. Excellent time management, adaptability, and effective written communication help individuals excel in this position. These skills and qualities ensure accurate and consistent evaluations of AI-generated content, directly impacting the improvement of artificial intelligence systems.

What does an AI Rater do?

A typical day for an AI Rater involves reviewing and evaluating various types of content, such as search engine results, social media posts, advertisements, or chatbot responses, to ensure they meet quality and relevancy standards. You may follow detailed guidelines to rate or annotate content, complete assigned tasks in a web-based platform, and provide feedback to help improve AI performance. Most positions are remote and offer flexible schedules, allowing you to plan your workload around personal commitments. Collaboration is generally limited, as most work is performed independently, but periodic communication with team leads for training or updates is common.

What are the most commonly searched types of Ai Rater jobs in California?

The most popular types of Ai Rater jobs in California are:

What cities in California are hiring for Ai Rater jobs?

Cities in California with the most Ai Rater job openings:

Infographic showing various Ai Rater job openings in California as of August 2026, with employment types broken down into 62% Full Time, 22% Part Time, 3% Temporary, and 13% Contract. Highlights an 82% In-person, and 18% Remote job distribution.

Senior Product Manager -- AI Agents

Rivio AI Inc.

San Francisco, CA • On-site

$150 - $210/hr

Other

Medical, Dental, Vision

Posted 10 days ago


Job description

San Francisco Hybrid Full-time Competitive salary + equity

Rivio is building the procurement intelligence platform that enterprise teams actually want to use. We're a small, high-output team and we're hiring our first PM who can own both the strategy and the work, without needing either defined for them.

You'll work directly with our CEO and engineering team to shape what gets built, drive clarity on the hardest problems, and make sure what ships actually works. The environment is genuinely ambiguous and undefined, and there's no existing playbook.

About Rivio

Rivio is a San Francisco-based AI technology company built for modern procurement teams that need to make better decisions, faster, and with confidence. Powered by Sheldon, our AI procurement employee, Rivio embeds world-class procurement data and expertise directly into sourcing and negotiation workflows, helping teams capture repeatable savings, expand coverage without adding headcount, and build institutional knowledge that compounds over time. Rivio helps in autonomously managing the tactical spend of an organization providing AI as a service.

Rivio works with customers such as Brex, Checkr, and Harvey, and is backed by Gradient Ventures, MIT, and S32, with strategic advisors from Meta, Apple, and other leading technology companies.

What You'll Do

On the product team, you'll own the roadmap for Sheldon and partner with customers and internal teams to ship features that make our platform smarter, faster, and more valuable to the procurement teams that depend on it. This is a unique opportunity to be the product manager of an AI employee, a multi-agent system designed to transform procurement organizations.

Think of this role as a mix of strategic and operational work:

Strategic (Product Sense)
  • Ownership to define the product roadmap based on the company's objective and customer feedback
  • Designing agent evaluation frameworks that give enterprise teams visibility into how Sheldon is performing and where it needs improvement including accuracy, task completion rate, and when Sheldon should elevate to a human
  • Define success metrics for the product
  • Stay plugged into the frontier of agentic AI such as new architectures, tooling, and techniques, and bring what's relevant back into Sheldon's roadmap
Operational (Product Execution)
  • Interview and analyze customer interactions to find insights on how to improve the product
  • Break down the roadmap into weekly sprints
  • Analyze bugs to diagnose issues, form clear hypotheses to explain anomalies, and prioritize solutions based on data rather than intuition
  • Navigate day-to-day PM responsibilities and respond quickly to unexpected product changes, staying grounded in metric tracking to guide decisions under ambiguity
  • Prototype rapidly using tools like Claude, Cursor, or v0 to validate ideas before committing engineering time

You'll partner closely with engineering, design, go-to-market, and our founders to make sure what we build solves real problems for the customers we have today and the ones we're going after next.

What We're Looking For
  • 5-8 years of product management experience, comfortable moving between strategic and operational work
  • You've owned a roadmap, driven prioritization under pressure, and shipped things real customers depend on
  • You've operated in genuinely undefined environments, building structure and requirements from scratch against constantly shifting parameters, not inheriting a mature roadmap and optimizing it
  • Experience building AI-native products, not just using AI in your workflow. You've shipped products where AI is the core of what customers are paying for
  • You find the root cause yourself before escalating. Engineering is your last call, not your first
  • Strong enough technical instincts to have a real opinion on architecture tradeoffs, spec out integrations, and evaluate whether an AI system is actually working
  • Strong communicator across engineering, customers, and leadership. You translate between what customers need, what engineering can build, and what the business should prioritize
Nice to Have
  • B2B product background preferred. Domain experience in procurement, supply chain, or finance is a bonus
  • Experience working directly with a founder as a primary product counterpart
  • Background in product operations alongside core PM work
What We Offer
  • Competitive salary and equity based on experience
  • Significant impact at an early-stage startup backed by top-tier investors
  • Direct access to the CEO and engineering leadership — no layers, no politics
  • Real path to Head of Product as the team scales
  • Comprehensive health insurance: medical, dental, and vision
What We Believe
  • Drive: We hire for drive first. It shows up as deep ownership, working on things no one asked you to do, and caring about outcomes, not just outputs.
  • Velocity: Ship at 70-80% confidence, iterate fast, refuse to wait for perfect conditions.
  • Agency: Act without permission. Set your own direction. Treat problems as experiments.
Why You Shouldn't Join
  • You need a defined roadmap and an established research function to feel productive
  • You're more comfortable managing process than making product decisions
  • You treat escalating to engineering as a first step, not a last resort
#J-18808-Ljbffr