Ai Rater Jobs in Chicago, IL (NOW HIRING)

The AI Bot Optimization Specialist owns the ongoing performance, testing, and refinement of Inspira's virtual agent experience. This role analyzes conversation-level and aggregate performance data, runs structured tests ahead of and following changes, and tunes the AI Bot configuration and conversational flows to improve resolution rate, deflection, and customer satisfaction. The Bot Optimization Specialist works in close partnership with the Content Specialist(s), translating performance data into prioritized content and configuration fixes, and shares accountability for the digital experience's overall self-service effectiveness.

• Own day-to-day performance monitoring, testing, and refinement of Inspira AI bot, including, but not limited to, Fin Voice where applicable, targeting resolution rate, deflection, and CSAT goals.
• Conduct weekly review of conversation transcripts, escalations, and handoffs to identify knowledge gaps, mis-routed intents, and breakdowns in bot understanding.
• Design and execute structured test scenarios, including regression and A/B testing of flows and prompts, prior to and following knowledge base or configuration changes.
• Tune AI bot's retrieval configuration, intents, and conversational flows based on performance data and testing outcomes.
• Partner with the Content Specialist to translate identified knowledge gaps into prioritized content tickets, and validate that published fixes resolve the underlying issue.
• Monitor and test configuration changes affecting AI bot's suggestions for human agents.
• Maintain a bot performance dashboard covering resolution rate, deflection rate, CSAT, and escalation volume, reporting trends to leadership on a monthly cadence.
• Lead root-cause analysis for significant bot performance dips or recurring failure patterns, driving remediation with the Content Specialist and platform/IT support as needed.
• Monitor AI Insights, including CX Score, Topics Explorer, and Trends reports, to identify emerging customer topics not yet covered by the bot.
• Make light, time-sensitive content corrections directly, such as typo fixes or quick clarifications, to resolve an active bot failure, coordinating promptly with the Content Specialist to formalize the change.
• Support new bot use case rollouts, including test plan development and go-live monitoring for new flows or digital agent capabilities.
• Stay current on AI platform updates and features, evaluating applicability to Inspira's use case.
• Perform other duties as assigned.

Education & Experience:

• 2+ years of experience in conversational AI or chatbot optimization, digital customer experience analytics, or CX systems administration; experience with Intercom Fin or a comparable AI virtual agent platform strongly preferred.
• Experience with contact center or digital CX performance metrics, including resolution rate, deflection rate, CSAT, and NPS, required.
• Familiarity with prompt engineering, RAG-based retrieval systems, or conversational flow design preferred.
• Experience with A/B testing, structured QA, or data-driven experimentation in a customer-facing digital tool preferred.
• Bachelor's degree in Business, CX, Data Analytics, Computer Science, or a related field preferred; equivalent experience considered.

Skills & Abilities:

• Strong analytical skills; able to interpret conversation-level and aggregate performance data to identify patterns and root causes.
• Detail-oriented and methodical approach to testing; able to design and execute structured test plans before deploying changes into a live customer-facing environment.
• Working understanding of how knowledge base content quality drives bot performance; able to translate bot failures into clear, actionable content or configuration requests.
• Comfortable partnering across content, platform administration, and compliance functions to resolve performance issues.
• Effective communicator; able to present bot performance trends and recommendations to non-technical leadership audiences.
• Comfortable operating in a fast-paced, iterative environment with frequent testing cycles and evolving platform capabilities.
• Demonstrates the highest degree of honesty, integrity, and ethical conduct in all professional activities.