Manager Ai Rater Jobs in Kentucky (NOW HIRING)

What you will own

• Strategy and roadmap:Define the vision, strategy, and roadmap for a complex AI product area. Decide what to build and why, prioritize across competing opportunities, and adapt as the model landscape and the business change.

• Business discovery:Partner directly with business units to map their operations end to end (service design), then translate ambiguous business problems into clearly scoped AI use cases with a clear value proposition, business case, and ROI justification.

• Hands-on prototyping:Build working prototypes anddemos yourselfusing agentic coding and AI tools (for example, Claude Code, Cursor, OpenAI Codex) to test feasibility and value beforecommittinga full team.

• Requirements and scoping:Turn validated use cases into clear, buildable requirements,user stories, acceptance criteria, edge cases, and success metrics,so engineers, datascientists, and designers know exactly what "done" and "good" mean. Shape the end-to-end user experience withdesign, sinceenterprise adoption lives or dies on usability.

• Delivery and execution:Drive the build from prototype to productionandown the backlog, sequence the work, make scope and tradeoff calls as models and constraints shift, unblockthe team, and keep momentum from first demo through launch and iteration.

• Evaluation and quality:Own how quality is defined and measured. Build golden sets and ground truth, run offline and live (production) evals, choose appropriate metrics (accuracy, precision, recall, F1, task success, hallucination rate), apply LLM-as-judge where it has been validated, and turn results into launch decisions and readouts for leadership.

• SupportSolution architecture:Supporthigh-level architecture and feasibilitydiscussionswith engineering, while owning the required prompt engineering and/or skill creation components of the AI harness.

• Cross-functional leadership:Lead engineers, data scientists, designers, and business partners without direct authority. Author reusable playbooks, skills, and solution patterns that make AI delivery faster and more consistent across the organization.

• Governed delivery:Move solutions through Humana's enterprise AI governance (AIRB, LRC, Responsible AI Council), owning theproductdocumentation,scorecards, and stage-gate reviews.

• Communication and mentorship:Communicate strategy,requirements,roadmap, ROI, andevalresults clearlyto executive and technical audiences. Mentor product managers and raise the team's standard for building with AI.

• Staying current:Track advances in models, agents, evals, and emerging techniques (agent harness, loop, and graph /GraphRAGapproaches) and apply them in your portfolio.

• B.S. or M.S. in Computer Science, Engineering, or a related field (or equivalent experience)

• 7+ years of product or related experience, including 4+ years in product management, with AI/ML or GenAI products shipped to production.

• Hands-on building:You use AI tools regularly and can articulate specifically what you would change about a model's behavior and why. You prototype your own ideas.

• Evaluation experience:You have built evals, including golden sets, ground truth, and offline and live evals, and used precision- and recall-based metrics to make product decisions.

• Systems thinking:When you find a problem, you build the infrastructure that prevents the whole class ofproblem.

• Communication across domains:You move between business and technical concepts, witha track recordof business cases, ROI models, and roadmaps that inform investment decisions.

• AI knowledge:A working understanding of modern AI/ML and GenAI (LLMs, agents, RAG, prompt engineering, andEvaluationmethodology) and how it applies to enterprise problems.

Preferred Qualifications

• Experience in healthcare, insurance, or another regulated industry.

• A well-supported point of view on where agentic AI, evals, and AI-native product development are headed.

• Hands-on experience with agent frameworks,GraphRAG/ knowledge graphs, and reusable skills / plugins.

• Comfort with ambiguity and a fast pace, working from first principles.