1

Ai Rater Jobs in Colorado (NOW HIRING)

AI Platform Operations Manager

Denver, CO ยท On-site

$54.25 - $74.25/hr

... rates, token consumption, and model endpoint performance. * Participate in on-call rotation, lead ... Work directly with AI engineers, data engineers, and enterprise application teams to remove ...

Job#: 3043651 AI Delivery Manager Location: Broomfield, Colorado (Partial Remote) Role Overview We ... Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You ...

Senior AI Automation Engineer

Denver, CO

$107K - $140K/yr

Job Title: Senior AI Automation Engineer Company: Prologis A day in the life The Senior AI ... Measure solution value through automation rate, adoption, accuracy, customer satisfaction ...

AI/ML Engineer

Aurora, CO ยท On-site

$82K - $172K/yr

AI/ML Engineer Job Category: Science Time Type: Full time Minimum Clearance Required to Start: None ... rates, relevant prior work experience, specific skills and competencies, education, and ...

AI/ML Engineer

Aurora, CO ยท Hybrid

$82K - $172K/yr

AI/ML Engineer Job Category: Science Time Type: Full time Minimum Clearance Required to Start: None ... rates, relevant prior work experience, specific skills and competencies, education, and ...

Showing results 21-40

Ai Rater information

What is an AI Rater?

An AI Rater evaluates and provides feedback on artificial intelligence models, typically improving search engines, chatbots, or recommendation systems. They assess the relevance, accuracy, and quality of AI-generated content based on specific guidelines. This role requires strong analytical skills, attention to detail, and familiarity with the subject matter being reviewed. AI Raters often work remotely and on a flexible schedule.

What does an AI Rater do?

A typical day for an AI Rater involves reviewing and evaluating various types of content, such as search engine results, social media posts, advertisements, or chatbot responses, to ensure they meet quality and relevancy standards. You may follow detailed guidelines to rate or annotate content, complete assigned tasks in a web-based platform, and provide feedback to help improve AI performance. Most positions are remote and offer flexible schedules, allowing you to plan your workload around personal commitments. Collaboration is generally limited, as most work is performed independently, but periodic communication with team leads for training or updates is common.

What skills and qualifications are needed to thrive as an AI Rater?

To thrive as an AI Rater, you generally need strong attention to detail, analytical thinking, and proficiency in English, often supported by formal education such as a high school diploma or higher. Familiarity with web browsers, online research, and company-specific rating platforms or guidelines is essential. Excellent time management, adaptability, and effective written communication help individuals excel in this position. These skills and qualities ensure accurate and consistent evaluations of AI-generated content, directly impacting the improvement of artificial intelligence systems.

How to become an AI Rater?

To become an AI Rater, candidates typically need a high school diploma or equivalent, strong language and analytical skills, and the ability to follow detailed guidelines. The role often involves evaluating search engine results or content for accuracy and relevance, using online platforms or specific assessment tools. Prior experience in data annotation or quality assurance can be beneficial, and flexible scheduling is common.

What are the most commonly searched types of Ai Rater jobs in Colorado?

The most popular types of Ai Rater jobs in Colorado are:

What cities in Colorado are hiring for Ai Rater jobs?

Cities in Colorado with the most Ai Rater job openings:

Infographic showing various Ai Rater job openings in Colorado as of September 2026, with employment types broken down into 53% Part Time, and 47% Contract. Highlights an 100% Remote job distribution.

AI Platform Operations Manager

Denver, CO โ€ข On-site

$54.25 - $74.25/hr

Full-time

Medical, Dental, Vision, Life, Retirement, PTO

Posted 15 days ago


Job description

THE COMPANY:

STACK INFRASTRUCTURE (STACK) provides digital infrastructure to scale the world's most innovative companies. We are an award-winning industry leader in building, owning, and operating highly efficient, cost-effective wholesale, colocation, and cloud data centers. Each of our national facilities meets or exceeds the highest industry standards in all operational categories of availability, security, connectivity, and physical resilience.

STACK offers the scale and geographic reach that rapidly growing hyperscale and enterprise companies need. The world runs on data. Data runs on STACK.

THE POSITION:

The DevOps Engineer, AI Platform is responsible for automating, deploying, and operating the infrastructure and delivery pipelines that support STACK's enterprise AI platform on Azure. This is a hands-on engineering role focused on build and run - not oversight.

Reporting to Head of AI, Enterprise AI & Data Strategy org this individual owns the infrastructure-as-code, CI/CD, containerization, observability, and release automation that allow AI engineers and enterprise application teams to ship agentic AI solutions, RAG pipelines, and integration services reliably and repeatably. The role sits at the intersection of cloud infrastructure, platform engineering, and MLOps - turning platform architecture into automated, governed, observable, and cost-efficient environments that teams across the organization build on.

KEY RESPONSIBILITIES

Infrastructure Automation & Infrastructure as Code

  • Build, maintain, and version infrastructure-as-code modules for Azure environments using Terraform, Bicep, or ARM, including compute, networking, storage, identity, and AI platform resources.

  • Automate provisioning of AI platform components - Azure AI Foundry, Azure OpenAI Service, Azure AI Search, Cosmos DB, ADLS Gen2, and Databricks - as reusable, parameterized deployment patterns.

  • Maintain environment parity across development, test, and production, including configuration management, drift detection, and remediation.

  • Implement and enforce tagging, naming, and resource organization standards that support governance, chargeback, and lifecycle management.

  • Automate routine platform operations - patching, certificate rotation, key and secret rotation, backup validation, and disaster recovery testing.

CI/CD & Release Engineering

  • Design, build, and operate CI/CD pipelines in Azure DevOps or GitHub Actions for application code, infrastructure code, container images, and AI/agent deployments.

  • Implement automated build, test, security scanning, artifact management, and promotion gates across environments.

  • Establish branching strategies, code review standards, and release management practices in partnership with AI engineering and enterprise application teams.

  • Build deployment automation for agentic AI services, MCP (Model Context Protocol) servers, and integration workloads running on Azure Container Apps and Azure Kubernetes Service (AKS).

  • Support model and prompt release workflows - versioning, staged rollout, evaluation gates, and rollback procedures for LLM-based applications.

Container Platform & AI Workload Operations

  • Operate and tune AKS and Azure Container Apps, including cluster upgrades, node pool sizing, autoscaling, ingress, networking, and workload isolation.

  • Build and maintain container images, base image standards, and registry governance in Azure Container Registry.

  • Manage compute scheduling and scaling for AI workloads, including GPU-backed and inference-heavy workloads where required.

  • Implement resiliency patterns - health probes, retries, throttling, quota management, and failover - for AI endpoints and integration services.

Observability, Reliability & Incident Response

  • Instrument platform and AI services with logging, metrics, tracing, and alerting using Azure Monitor, Log Analytics, Application Insights, and equivalent open-source tooling.

  • Build dashboards and service-level indicators covering platform availability, latency, throughput, error rates, token consumption, and model endpoint performance.

  • Participate in on-call rotation, lead incident triage and resolution for platform issues, and drive root cause analysis and corrective actions.

  • Develop and maintain runbooks, operational documentation, and automated remediation for recurring issues.

Security, Governance & Cost Optimization

  • Implement DevSecOps practices - secrets management in Azure Key Vault, managed identity usage, least-privilege access, dependency and container vulnerability scanning, and policy-as-code.

  • Partner with Information Security to ensure pipelines and environments meet enterprise security, data residency, and compliance requirements.

  • Support Azure FinOps practices through cost visibility, rightsizing, reserved capacity, and automated controls on non-production and idle resources.

  • Maintain audit trails and change records for infrastructure and release activity.

Delivery & Cross-Functional Collaboration

  • Work directly with AI engineers, data engineers, and enterprise application teams to remove deployment friction and improve time-to-production for AI solutions.

  • Translate platform architecture and standards into automated, self-service capabilities that teams can consume without deep infrastructure knowledge.

  • Contribute to platform engineering standards, reference implementations, and internal documentation.

  • Provide technical escalation support for build, deployment, and environment issues.

THE DETAILS:

  • Location: Denver, CO

  • Travel: <10%

  • Benefits: Healthcare, Dental Care, Vision Insurance, Life Insurance, Paid Time Off, and Paid Leave Programs

  • Must be eligible to work in the United States

  • Must pass comprehensive background and drug screening

MUST-HAVE QUALIFICATIONS:

  • Bachelor's degree in Computer Science, Information Technology, Engineering, or related field, or equivalent practical experience.

  • 5+ years of hands-on experience in DevOps, site reliability engineering, or platform engineering roles with a strong delivery track record.

  • Strong proficiency in Infrastructure as Code - Terraform, Bicep, or ARM - including module design, state management, and reusable patterns.

  • Proven experience building and operating CI/CD pipelines in Azure DevOps or GitHub Actions.

  • Hands-on experience with containerization and orchestration - Docker, Azure Kubernetes Service (AKS), and Azure Container Apps or equivalent.

  • Solid working knowledge of Azure core services - compute, networking (VNet, NSG, Private Endpoints), storage, identity (Entra ID), and Key Vault.

  • Strong scripting and automation skills in Python, PowerShell, or Bash.

  • Experience with monitoring and observability tooling - Azure Monitor, Log Analytics, Application Insights, Prometheus, or Grafana.

  • Working knowledge of Git-based workflows, code review practices, and artifact/registry management.

  • Demonstrated ability to troubleshoot production issues across infrastructure, network, and application layers.

PREFERRED QUALIFICATIONS:

  • Microsoft Certified: DevOps Engineer Expert (AZ-400), Azure Administrator (AZ-104), or Certified Kubernetes Administrator (CKA).

  • Experience deploying and operating AI/ML workloads - model endpoints, RAG pipelines, vector databases, or agentic services in production.

  • Familiarity with MLOps tooling and practices - Azure Machine Learning, MLflow, Databricks, or equivalent model lifecycle platforms.

  • Experience deploying MCP (Model Context Protocol) servers or similar integration services connecting AI agents to enterprise systems.

  • Exposure to agentic AI frameworks such as Semantic Kernel, LangGraph, or AutoGen from a deployment and operations perspective.

  • Experience with GPU compute provisioning, quota management, and inference cost optimization.

  • Knowledge of FinOps frameworks and Azure cost optimization practices.

  • Experience integrating with enterprise systems such as Microsoft 365, Freshworks ITSM, Workday, NetSuite, or Procore.

  • Experience in data center, hyperscale, or infrastructure-intensive industry environments.

Compensation Range:

$128,260.00 - $146,017.59

THIS MIGHT BE RIGHT FOR YOU IF:

  • You are a strong communicator, you are persuasive and clear, blending analytics with experience in decision-making.

  • You do not get flustered easily. You can juggle multiple priorities while balancing urgent requests with shifting timelines and deliverables.

  • You are a team builder. You take the time to understand and develop the strengths of your resources while formulating long-term plans for the growth and success of the team.

  • You are naturally curious and driven toward continual improvement. While you celebrate your successes, you take time to review and analyze campaigns for future learning.

WHY STACK?

  • We offer a competitive compensation package with strong benefits, including medical, dental, and vision insurance, a 401K program, flexible spending accounts - even a cell phone subsidy.

  • We foster a culture of appreciation, including peer-to-peer recognition and rewards programs.

  • Fun is part of our DNA, with events, game nights, happy hours, and barbecues.

  • We're growing - this is a great time to join and make an impact!

STACK is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity and expression, age, national origin, mental or physical disability, genetic information, veteran status, or any other status protected by federal, state, or local law

Note to external agencies: We are not accepting any blind submissions or resumes/cvs from recruitment agencies. Any candidates sent to STACK Infrastructure, Inc. will not be accepted or considered as a submission without a signed agreement in place. Fees will not be paid in the event a candidate submitted by a recruiter without an agreement in place is hired; such resumes will be deemed the sole property of STACK Infrastructure, Inc.