ZipRecruiter

Log In

2

Remote Ai Rating Jobs (NOW HIRING)

Worth AI

Senior Agentic (AI) Engineer

Orlando, FL · On-site +1

$97K - $134K/yr

Measurable improvements in task success rate, grounding accuracy, and hallucination rate on our ... All Remote Hires will be required to travel to Orlando, Florida at least twice per year for Town ...

Weekday AI

Legal AI Specialist - Litigation

$85 - $120/hr

Fully remote with flexible scheduling * Open to candidates based in the United States, Canada, or ... Competitive hourly rate: $85-$120 per hour * Weekly payments via supported global payment platforms ...

Weekday AI

Investment Banking AI Specialist

$100 - $130/hr

... AI lab focused on foundational models. In this role, you will apply your expertise in financial ... Fully remote with flexible, self-managed schedule Compensation * Hourly rate: $100-$130 per hour

Weekday AI

Private Markets AI Specialist

$130/hr

... leading AI research lab focused on foundational models. In this role, you will apply your ... Fully remote with flexible, self-managed schedule Compensation * Starting rate: $130 per hour

Weekday AI

Insurance Claims Management AI Expert

$100 - $120/hr

US, UK, Canada, France, Portugal (remote) We are seeking a highly motivated and detail-oriented ... Implement AI tools to track recovery potential, automate documentation, and improve recovery rates ...

Welo Data

AI Content Writer - Remote

Columbia, SC · Remote

$14.50/hr

What Does a Search Quality Rater Do? (youtube.com) Why this work matters: Your feedback helps train ... Schedule & Support: Fully remote position Set your own schedule and complete tasks when it ...

Welo Data

AI Content Writer - Remote

Huntington, WV · Remote

$14.50/hr

What Does a Search Quality Rater Do? (youtube.com) Why this work matters: Your feedback helps train ... Schedule & Support: Fully remote position Set your own schedule and complete tasks when it ...

Welo Data

AI Content Writer - Remote

Fishers, IN · Remote

$14.50/hr

What Does a Search Quality Rater Do? (youtube.com) Why this work matters: Your feedback helps train ... Schedule & Support: Fully remote position Set your own schedule and complete tasks when it ...

Welo Data

AI Content Writer - Remote

Macon, GA · Remote

$14.50/hr

What Does a Search Quality Rater Do? (youtube.com) Why this work matters: Your feedback helps train ... Schedule & Support: Fully remote position Set your own schedule and complete tasks when it ...

Welo Data

AI Content Writer - Remote

Charleston, SC · Remote

$14.50/hr

What Does a Search Quality Rater Do? (youtube.com) Why this work matters: Your feedback helps train ... Schedule & Support: Fully remote position Set your own schedule and complete tasks when it ...

Welo Data

AI Content Writer - Remote

Chattanooga, TN · Remote

$14.50/hr

What Does a Search Quality Rater Do? (youtube.com) Why this work matters: Your feedback helps train ... Schedule & Support: Fully remote position Set your own schedule and complete tasks when it ...

Welo Data

AI Content Writer - Remote

Pembroke Pines, FL · Remote

$14.50/hr

What Does a Search Quality Rater Do? (youtube.com) Why this work matters: Your feedback helps train ... Schedule & Support: Fully remote position Set your own schedule and complete tasks when it ...

Welo Data

AI Content Writer - Remote

College Station, TX · Remote

$14.50/hr

What Does a Search Quality Rater Do? (youtube.com) Why this work matters: Your feedback helps train ... Schedule & Support: Fully remote position Set your own schedule and complete tasks when it ...

Welo Data

AI Content Writer - Remote

Saint George, UT · Remote

$14.50/hr

What Does a Search Quality Rater Do? (youtube.com) Why this work matters: Your feedback helps train ... Schedule & Support: Fully remote position Set your own schedule and complete tasks when it ...

next page

Showing results 1-20

All JobsAi Rating JobsRemote Ai Rating Jobs

Remote Ai Rating information

What is the difference between Remote Ai Rating vs Data Analyst?

AspectRemote Ai RatingData Analyst
Required CredentialsTypically requires AI, machine learning, or data science certificationsRequires degrees in statistics, mathematics, or related fields
Work EnvironmentRemote, often project-based with AI/tech companiesRemote or on-site, across various industries
Industry UsagePrimarily in AI, tech, and data-driven companiesAcross finance, healthcare, marketing, and more
Common Search/ComparisonYesYes

Remote Ai Rating and Data Analyst roles share similarities in requiring analytical skills and remote work options. However, Remote Ai Rating focuses more on AI-specific evaluation and machine learning knowledge, while Data Analysts handle broader data interpretation and reporting across industries.

What are the key skills and qualifications needed to thrive as a Remote AI Rater, and why are they important?

To thrive as a Remote AI Rater, you need strong analytical skills, attention to detail, and a good command of written English, often supported by at least a high school diploma or equivalent. Familiarity with web browsers, online research tools, and proprietary rating platforms is typically required, with some employers providing specific training or guidelines. Excellent time management, self-motivation, and clear communication are valuable soft skills for succeeding in a remote setting. These competencies ensure accurate and consistent evaluation of AI outputs, which is crucial for improving the quality and reliability of AI systems.

What are some common challenges faced by remote AI raters, and how can they be addressed?

Remote AI raters often face challenges such as maintaining focus during repetitive evaluation tasks, managing time effectively without direct supervision, and keeping up with changing guidelines. To address these, it's helpful to establish a structured daily routine, take regular breaks to avoid fatigue, and actively participate in team communications or training sessions. Staying organized and seeking clarification when guidelines change can also help ensure consistent, high-quality ratings while working remotely.

What is a Remote AI Rater?

A Remote AI Rater is a professional who evaluates and rates the quality, relevance, and accuracy of AI-generated content, such as search engine results, chatbot responses, or advertisements. They work from home and follow specific guidelines to assess whether AI outputs meet certain standards. Remote AI Raters help improve artificial intelligence systems by providing human feedback, ensuring that automated responses better serve users. This role typically requires attention to detail, good communication skills, and the ability to follow detailed instructions.
More about Remote Ai Rating jobs
What cities are hiring for Remote Ai Rating jobs? Cities with the most Remote Ai Rating job openings:
What are the most commonly searched types of Ai Rating jobs? The most popular types of Ai Rating jobs are:
What states have the most Remote Ai Rating jobs? States with the most job openings for Remote Ai Rating jobs include:
What job categories do people searching Remote Ai Rating jobs look for? The top searched job categories for Remote Ai Rating jobs are:
Remote Ai Rating jobs near you
Infographic showing various Remote Ai Rating job openings in the United States as of May 2026, with employment types broken down into 65% Full Time, 21% Part Time, and 14% Contract. Highlights an 100% Remote job distribution.

Senior Agentic (AI) Engineer

Worth AI

Orlando, FL • On-site, Remote

$97K - $134K/yr

Full-time

Medical, Dental, Vision, Life, Retirement, PTO

Posted 25 days ago

Job description

Worth AI is hiring a Senior Agentic AI Engineer to design and ship production agent systems that automate KYB, underwriting, and risk decisions on regulated financial data. You'll own agents end-to-end architecture, retrieval, tools, evals, and production deployment and partner closely with our Chief AI Officer, applied scientists, and platform teams.

Responsibilities
  • Design and ship multi-step agentic systems (planner/executor, tool-using, multi-agent, human-in-the-loop) for onboarding, underwriting, case review, and continuous monitoring.
  • Architect agent graphs in LangGraph (or comparable - CrewAI, AutoGen, Claude Agent SDK) with explicit state, durable execution, retries, and safe fallbacks.
  • Build the retrieval layer powering our agents - chunking, hybrid search, reranking, and grounded citation.
  • Own the eval stack: golden sets, offline regression suites, LLM-as-judge, online A/B and shadow evals, and red-teaming for jailbreaks, prompt injection, and PII leakage.
  • Expose agents to production systems via well-typed tools and MCP servers. Treat tool surface area as a product.
  • Drive production MLOps: deployment, versioning, traffic shaping, cost/latency budgets, tracing, and on-call playbooks for agent incidents.
  • Partner with security and compliance to keep agents inside SOC 2, GDPR, CCPA, and fair-lending posture - auditability and explainability built in, not bolted on.
  • Mentor engineers on agent patterns, prompt hygiene, eval discipline, and LLM failure modes.
  • Technology Stack
    • Languages: Python, Node.js, TypeScript
    • Agent / LLM frameworks: LangGraph, LangChain, Claude Agent SDK, MCP, OpenAI SDK
    • Models: Anthropic Claude, OpenAI, open-weight where appropriate
    • Retrieval & Data: PostgreSQL, pgvector, OpenSearch, Kafka, Redshift, Redis
    • Infra: AWS, Kubernetes (EKS), ArgoCD, Terraform
    • Evals & Observability: LangSmith / Langfuse / Braintrust-style tooling, DataDog

Requirements

  • 5+ years of software engineering experience, with 2+ years building production LLM or agentic systems (not just notebooks or demos).
  • Hands-on experience with a modern agent framework (LangGraph strongly preferred) and a track record of shipping agents that run, fail gracefully, and recover.
  • Strong RAG fundamentals chunking, embeddings, hybrid retrieval, reranking, grounding - and judgment about when RAG isn't the right answer.
  • Real eval experience golden sets, offline and online evaluations, used to make ship/no-ship calls.
  • Production MLOps fluency: deployed LLM workloads under real latency, cost, and reliability constraints.
  • Strong Python; comfortable in TypeScript / Node.js.
  • Solid systems engineering instincts APIs, async patterns, queues, databases, distributed system failure modes.
  • Calibrated communicator; thrives in ambiguous, fast-moving environments.
  • Prior experience in fintech, lending, payments, KYB/KYC, fraud, or AML.
  • Experience building MCP servers or other structured tool interfaces for LLMs.
  • Background in classical ML (ranking, scoring, calibration).
  • Experience designing explainable / auditable AI workflows for regulated environments.
  • Open-source contributions to agent frameworks, eval tooling, or retrieval libraries.
  • AWS depth (EKS, MSK, RDS, S3, Lambda) and IaC with Terraform.
Success Metrics
  • Agent Quality: Measurable improvements in task success rate, grounding accuracy, and hallucination rate on our eval suites.
  • Production Reliability: Agents you own meet defined SLOs for latency (P90/P99), tool-call success, and cost per task.
  • Velocity: New agent capabilities go from prototype to production in weeks, without skipping evals or guardrails.
  • Risk Posture: Zero material incidents tied to prompt injection, PII leakage, or unsafe tool use on agents you own.
  • Force Multiplier: Patterns, tools, and eval scaffolding you build get adopted across engineering.

All Remote Hires will be required to travel to Orlando, Florida at least twice per year for Town Halls and team collaboration, in addition to orientation in Orlando.

Benefits

  • Health Care Plan (Medical, Dental & Vision)
  • Retirement Plan (401k, IRA)
  • Life Insurance
  • Flexible Paid Time Off
  • 9 paid Holidays
  • Family Leave
  • Remote
  • Hybrid work (for Orlando Associates)
  • Free Food & Snacks (Orlando)
  • Wellness Resources

Apply