ZipRecruiter

Log In

1

Ai Rater Jobs in Decatur, GA (NOW HIRING)

Florence Healthcare

Senior AI Engineer

Atlanta, GA · On-site

$100K - $138K/yr

... rate, error propagation) • Debug and optimize agent decision-making and tool selection behavior • Experience building and deploying production-grade systems • Ability to debug complex issues ...

Florence Healthcare - US

Senior AI Engineer

Atlanta, GA · On-site

$100K - $138K/yr

We are looking for a Senior AI Engineer to design, build, and deploy high-quality AI-powered ... Evaluate agent performance across multi-step tasks (task success rate, error propagation) * Debug ...

DataAnnotation

Treasury Analyst - AI Trainer

Atlanta, GA · Remote

$50 - $60/hr

Projects are paid hourly starting at USD $50-$60 per hour, with bonus rates available on some projects Responsibilities: * Give AI chatbots diverse and complex problems and evaluate their outputs

DataAnnotation

Financial Planner - AI Trainer

Atlanta, GA · Remote

$50 - $60/hr

Projects are paid hourly starting at USD $50-$60 per hour, with bonus rates available on some projects Responsibilities: * Give AI chatbots diverse and complex problems and evaluate their outputs

Apply Early

DataAnnotation

DevSecOps Architect - AI Trainer

Atlanta, GA · Remote

$50 - $100/hr

Projects are paid hourly starting at $50-100+/hr, with bonus rates available on some projects * Impact: help shape the future of AI technologies built for cybersecurity. Responsibilities: * Evaluate ...

Apply Early

DataAnnotation

Penetration Tester - AI Trainer

Atlanta, GA · Remote

$50 - $100/hr

Projects are paid hourly starting at $50-100+/hr, with bonus rates available on some projects * Impact: help shape the future of AI technologies built for cybersecurity. Responsibilities: * Evaluate ...

Apply Early

DataAnnotation

SOC Analyst - AI Trainer

Sandy Springs, GA · Remote

$50 - $100/hr

Projects are paid hourly starting at $50-100+/hr, with bonus rates available on some projects * Impact: help shape the future of AI technologies built for cybersecurity. Responsibilities: * Evaluate ...

Apply Early

next page

Showing results 1-20

All JobsAi Rater JobsAi Rater Jobs in Decatur, GA

Ai Rater information

What is an AI Rater job?

An AI Rater evaluates and provides feedback on artificial intelligence models, typically improving search engines, chatbots, or recommendation systems. They assess the relevance, accuracy, and quality of AI-generated content based on specific guidelines. This role requires strong analytical skills, attention to detail, and familiarity with the subject matter being reviewed. AI Raters often work remotely and on a flexible schedule.

What are the key skills and qualifications needed to thrive in the Ai Rater position, and why are they important?

To thrive as an AI Rater, you generally need strong attention to detail, analytical thinking, and proficiency in English, often supported by formal education such as a high school diploma or higher. Familiarity with web browsers, online research, and company-specific rating platforms or guidelines is essential. Excellent time management, adaptability, and effective written communication help individuals excel in this position. These skills and qualities ensure accurate and consistent evaluations of AI-generated content, directly impacting the improvement of artificial intelligence systems.

What does a typical day look like for an AI Rater?

A typical day for an AI Rater involves reviewing and evaluating various types of content, such as search engine results, social media posts, advertisements, or chatbot responses, to ensure they meet quality and relevancy standards. You may follow detailed guidelines to rate or annotate content, complete assigned tasks in a web-based platform, and provide feedback to help improve AI performance. Most positions are remote and offer flexible schedules, allowing you to plan your workload around personal commitments. Collaboration is generally limited, as most work is performed independently, but periodic communication with team leads for training or updates is common.

What are popular job titles related to Ai Rater jobs in Decatur, GA? For Ai Rater jobs in Decatur, GA, the most frequently searched job titles are:
What job categories do people searching Ai Rater jobs in Decatur, GA look for? The top searched job categories for Ai Rater jobs in Decatur, GA are:
What cities near Decatur, GA are hiring for Ai Rater jobs? Cities near Decatur, GA with the most Ai Rater job openings:
Ai Rater jobs near you
Senior AI Engineer

Senior AI Engineer

Florence Healthcare

Atlanta, GA • On-site

$100K - $138K/yr

Full-time

Posted 28 days ago

Job description

Job Summary:
Florence Healthcare is a company that advances cures by helping research sites improve their workflow. They are seeking a Senior AI Engineer to design, build, and deploy AI-powered features, focusing on end-to-end implementation of AI systems while collaborating with product and engineering teams.
Responsibilities:
• Design and implement AI-powered features (LLM workflows, copilots, and agent-based systems with tool use and multi-step reasoning)
• Own the full lifecycle: prototyping → evaluation → production deployment → iteration
• Ensure solutions are reliable, performant, and aligned with product needs
• Build and optimize:
• Prompt pipelines for specific use cases
• Retrieval systems (embeddings, chunking, ranking)
• RAG-based workflows where needed
• Iterate on outputs to improve quality, accuracy, and consistency
• Design scalable and cost-efficient AI architectures for production workloads
• Select and evaluate models (hosted vs open-source) based on use case constraints
• Design and build agentic workflows capable of multi-step reasoning and decision-making
• Integrate agents with tools, APIs, and internal systems to perform real-world actions
• Implement planning, execution, and reflection loops for complex tasks
• Manage context, memory, and state across multi-step interactions
• Balance deterministic workflows vs. agent autonomy for reliability and control
• Run structured experiments to compare approaches (prompting, retrieval, models)
• Define and track key metrics for AI performance (quality, latency, cost)
• Debug and improve non-deterministic system behavior
• Build and maintain evaluation datasets and benchmarks
• Implement automated evaluation pipelines for continuous improvement
• Drive technical direction and influence AI adoption across teams
• Partner with product managers and designers to scope AI features
• Contribute to shared patterns and reusable components
• Participate in code reviews and design discussions
• Support and mentor mid-level engineers where needed
• Design guardrails to ensure safe and reliable AI behavior
• Mitigate hallucinations, prompt injection, and model misuse
• Ensure compliance with data privacy and enterprise requirements
• Implement monitoring and observability for AI systems in production
• Implement guardrails for agent actions (tool access control, execution boundaries)
• Prevent failure cascades in multi-step agent workflows
Qualifications:
Required:
• Strong understanding of LLM capabilities and limitations
• Experience with prompt engineering and structured output design
• Hands-on experience with embeddings and vector search
• Familiarity with RAG architectures and when to apply them
• Experience designing agent-based architectures (AgentCore concepts)
• Understanding of tool use, planning strategies, and memory mechanisms in LLM systems
• 4+ years of related work experience
• Solid backend/system design fundamentals
• Evaluate agent performance across multi-step tasks (task success rate, error propagation)
• Debug and optimize agent decision-making and tool selection behavior
• Experience building and deploying production-grade systems
• Ability to debug complex issues, including probabilistic outputs
• Comfort working with APIs, pipelines, and data flows
• Ability to translate user needs into effective AI solutions
• Strong intuition for balancing quality, latency, and cost
• Focus on delivering measurable product impact
• Communicates clearly across engineering and product teams
• Contributes to team knowledge and shared practices
Company:
Florence is a clinical trials platform that connects pharmaceutical companies and study sites to decentralize trials for drug development. Founded in 2014, the company is headquartered in Atlanta, USA, with a team of 201-500 employees. The company is currently Growth Stage.

Apply