ZipRecruiter

Log In

2

Remote Search Quality Rater Jobs in Minnesota (NOW HIRING)

UnitedHealth Group

Senior AI ML Engineer - Remote

Eden Prairie, MN · On-site +1

$124K - $164K/yr

  • Retirement

... enabling high-quality, fully accountable, value-based care as part of Optum's mission to help ... Build systems for document ingestion, chunking, embeddings, semantic search, and advanced retrieval ...

SCHWAN'S

Strategic Sourcing Buyer of EHS

Marshall, MN · On-site +1

$68K - $113K/yr

  • Medical

  • Retirement

Troubleshoot and resolve quality, cost, and delivery (QCD) issues with internal and external ... Drive results in price, rate, and risk management while maximizing supplier value and reliability.

SCHWAN'S

Capital Buyer

Marshall, MN · On-site +1

$68K - $113K/yr

  • Medical

  • Retirement

Troubleshoot and resolve quality, cost, and delivery (QCD) issues with internal and external ... Drive results in price, rate, and risk management while maximizing supplier value and reliability.

HomeServices of America

Marketing Manager

Eden Prairie, MN · On-site +1

$87K - $118K/yr

  • Medical

  • Dental

  • Vision

  • Life

  • Retirement

  • PTO

Maintain online search advertising campaigns. 2. Work with sales and executive management to ... This position is open to remote work hiring within AZ, CO, KY, MN, MS, NC, PA or TX, or if the ...

HomeServices of America

Marketing Manager

Eden Prairie, MN · On-site +1

$87K - $118K/yr

  • Medical

  • Dental

  • Vision

  • Life

  • Retirement

  • PTO

Maintain online search advertising campaigns. 2. Work with sales and executive management to ... This position is open to remote work hiring within AZ, CO, KY, MN, MS, NC, PA or TX, or if the ...

Steele Recruiting

Executive Assistant to the COO

Litchfield, MN · Remote

  • Medical

  • Dental

  • Vision

  • Life

  • Retirement

  • PTO

... the search for an Executive Assistant to the COO. This is a fully remote job, supporting an ... quality of life. Capstone serves as a fiduciary and takes its responsibility to act in clients ...

Tesoro Group

Account Manager

Minneapolis, MN · Remote

The Dallas - Remote Account Manager will seek out and sign new client relationships, manage ... The Account Manager will develop, retain, and grow client relationships to ensure high-quality ...

previous page

Showing results 41-60

All JobsSearch Quality Rater JobsRemote Search Quality Rater JobsRemote Search Quality Rater Jobs in Minnesota

Remote Search Quality Rater information

What challenges do remote search quality raters face when evaluating search engine results?

Remote Search Quality Raters often encounter challenges such as interpreting ambiguous queries, staying updated on evolving search guidelines, and ensuring consistency across diverse types of content. Since raters work independently and remotely, time management and self-motivation are crucial to maintain productivity. Additionally, balancing speed with accuracy is essential, as raters are expected to meet quality benchmarks while handling a variety of tasks each day.

What is a remote search quality rater?

Remote Search Quality Raters are individuals hired to evaluate the quality and relevance of search engine results. They work from home, typically on a flexible schedule, and use specific guidelines provided by search engine companies to assess whether search results meet user needs. Their feedback helps improve search algorithms and overall user experience. This role does not involve coding or direct interaction with users, but rather focuses on analyzing and rating search queries, web pages, and advertisements.

What skills and qualifications are needed to be a remote search quality rater?

To excel as a Remote Search Quality Rater, you need strong analytical skills, attention to detail, and proficiency in web research, typically requiring at least a high school diploma or equivalent. Familiarity with search engines, web browsers, and company-specific rating guidelines is crucial, as well as the ability to quickly learn proprietary evaluation platforms. Excellent communication, time management, and self-motivation are important soft skills for meeting productivity targets while working independently. These competencies ensure accurate and consistent evaluations, which are critical for improving search engine algorithms and user experience.
What are popular job titles related to Remote Search Quality Rater jobs in Minnesota? For Remote Search Quality Rater jobs in Minnesota, the most frequently searched job titles are:
What job categories do people searching Remote Search Quality Rater jobs in Minnesota look for? The top searched job categories for Remote Search Quality Rater jobs in Minnesota are:
What cities in Minnesota are hiring for Remote Search Quality Rater jobs? Cities in Minnesota with the most Remote Search Quality Rater job openings:
Remote Search Quality Rater jobs near you
Infographic showing various Remote Search Quality Rater job openings in Minnesota as of August 2026, with employment types broken down into 40% Full Time, and 60% Part Time. Highlights an 100% Remote job distribution.

Senior AI Engineer - Agentic Systems & Data Pipelines

Collaboration.Ai

Minneapolis, MN • On-site, Remote

$110K - $150K/yr

Full-time

Posted 15 days ago

Job description

Who We Are
Collaboration.Ai is a mission-focused, AI-powered software and services company based in Minnesota, with employees, partners, and customers around the world. We unite people, technology, and purpose to accelerate breakthroughs that transform industries, empower communities, and create a more sustainable future. We collaborate with organizations across the defense ecosystem, helping them navigate complex challenges and drive transformative change.
Our Products
NetworkOS - NetworkOS is an AI-powered platform that aligns people, purpose, ideas, and expertise in real-time, generating actionable insights to propel movements forward.
CrowdVector - CrowdVector is an integrated solution marketplace and innovation management platform that rapidly uncovers new ideas and advances breakthroughs to fuel movements.
To learn more about us, visit collaboration.ai.
About the Role
You'll build the agentic systems and data pipelines behind NetworkOS's AI capabilities: production agent workflows built on industry-leading agent SDKs and harnesses, MCP servers, and Agent Skills standards; the eval and observability layer that keeps LLM quality measurable; and the ingestion pipelines that turn messy, diverse data sources into queryable knowledge.
This is an execution seat, not an ivory tower. You'll commit code every week, ship agents as product capability rather than demos, and help shape a roadmap that's heading deep into graph + agents territory - for customers in defense, healthcare, and regulated enterprise.
Agents in production. Pipelines that hold. Evals that keep everyone honest.
This opportunity is remote with a preference for candidates in the Twin Cities area (Minneapolis, Saint Paul); however all candidates are encouraged to apply!
What You'll Do
  • Ship production agent systems - design, build, and operate agentic workflows (agent SDKs, MCP servers, Agent Skills standards) powering AI-driven matching, analysis, and data intelligence
  • Operationalize LLM quality - build the eval and observability layer with Langfuse, golden datasets, LLM-as-judge patterns, and FinOps-style tracking so every workflow has measurable quality, cost, and latency
  • Engineer data pipelines - robust ingestion of documents, structured data, and external sources into searchable knowledge bases with quality validation, deduplication, and incremental updates
  • Own retrieval quality - hybrid search combining vector, keyword, and metadata retrieval, continuously improved through reranking, query expansion, and contextual compression
  • Accelerate with AI - build custom MCP tools and Agent Skills that make the whole engineering team measurably faster
  • Execute alongside the team - pair with full-stack engineers on AI integration points, contribute to incident response for AI services, and keep your hands in the code
Our Tech Stack
  • Languages: Python (primary); Kotlin (core platform language at CAI); TypeScript/Node.js and other modern languages (secondary)
  • AI/ML: FastAPI, Pydantic; multi-provider LLM SDKs (Anthropic, OpenAI, and others)
  • Agentic Tooling: Claude Code/Codex/etc.; industry-leading agent SDKs and harnesses; MCP servers; Agent Skills standards
  • LLM Operations: Langfuse + evals (golden datasets, LLM-as-judge); in-house FinOps tracking (token usage, latency, cost); multi-provider orchestration including AWS Bedrock
  • Search & Retrieval: Vector databases, OpenSearch, embedding models
  • Data: PostgreSQL, Amazon S3; streaming pipelines (Kafka/Kinesis) where needed
  • Infrastructure: Docker, Kubernetes (AWS EKS); DataDog + OpenTelemetry observability
What We're Looking For
Must Haves
  • 7+ years of professional software engineering experience, with 3+ years focused on AI/ML or data engineering
  • Production agentic/LLM application experience - built and operated systems around LLM APIs (Anthropic, OpenAI) serving real users: agents, tool-use, or orchestrated LLM workflows
  • Data engineering background - robust, scalable pipelines for AI/ML workloads
  • LLM operations experience - evals and observability for production LLM systems (quality, cost, latency)
  • Production retrieval experience - vector databases and/or search engines (OpenSearch, Elasticsearch)
  • Modern Python stack proficiency - FastAPI, Pydantic, async/await, modern dependency management
  • AI-native workflows - demonstrated ability to leverage Claude Code/Codex or similar agentic coding tools to accelerate development
  • Experience with Docker, Kubernetes, and AWS
  • US citizenship required (DoD contracting - IL4/IL5 environments - and FedRAMP compliance)
Nice-to-Haves
  • Deep agentic ecosystem experience - Agent Skills standards, custom MCP servers, agent SDKs across major vendors
  • Advanced RAG expertise - GraphRAG, agentic RAG, contextual retrieval, reranking strategies
  • Graph data experience - knowledge graphs, graph databases, or graph-based retrieval
  • Model selection & rightsizing - matching models to domain-specific use cases across quality, cost, and latency tradeoffs
  • Streaming data experience (Kafka, Kinesis) for real-time knowledge base updates
  • Research background, open-source contributions, or an advanced degree in ML/IR/NLP
Why Join Collaboration AI?
Real AI engineering, not a wrapper shop. Production agents, hybrid retrieval, continuous evals, and a roadmap heading into graph + agents - with the autonomy to shape how it's built.
AI-native by default. We build with AI, not just for AI. Agentic coding tools (Claude Code/Codex/etc.), agent SDKs and harnesses, MCP servers, and Agent Skills standards are how we work daily - you'll both use and build them.
Work that matters. Defense, healthcare, and regulated industries - SOC 2 and NIST compliance, FedRAMP readiness, and customers whose missions demand AI they can trust.
Small, senior team. Early-stage impact with your work visible from week one. You'll help set the bar for how AI engineering is done here.

Apply