Manager Ai Rater Jobs in Wisconsin (NOW HIRING)

Job Summary:
Acuity Insurance is seeking a Senior Software Engineer to design, build, and maintain production-ready Generative AI capabilities within existing applications and platforms. The role involves delivering secure, scalable AI solutions, collaborating with various stakeholders, troubleshooting production issues, and mentoring team members in best practices.
Responsibilities:
• Partner with business analysts, engineers, and stakeholders to understand business challenges and design strong AI solutions, including LLM based and agentic systems.
• Demonstrate professionalism and integrity while serving as a role model for engineers and promoting a culture of excellence and curiosity.
• Lead planning and scoping of AI projects, providing input on timelines, model selection, infrastructure, and deliverables.
• Develop and enhance AI applications including RAG pipelines, model evaluation frameworks, and agent orchestrations.
• Write clean, scalable, and maintainable code that supports high performing AI systems and meets reliability expectations.
• Demonstrate expert understanding of data ecosystems and how they support AI workflows including data vectorization, retrieval, and context routing.
• Perform testing, debugging, and output validation to ensure AI responses are accurate, safe, and aligned with business logic.
• Produce and maintain clear technical documentation including integration specs, system diagrams, integration guides, and operational runbooks.
• Identify and correct inefficiencies in code, architecture, prompts, data flows, and model parameter tuning.
• Mentor other engineers in AI best practices, model behavior, prompt design, and responsible AI principles.
• Perform code reviews and provide recommendations for accuracy, performance, and adherence to AI safety guidelines.
• Direct and manage AI projects from initiation through delivery including model selection, benchmarking, deployment, and monitoring.
• Identify and develop solutions for highly complex AI integration challenges like prompt chaining, retrieval design, agent coordination, and enterprise scale integrations.
• Understand the architecture of the applications and AI systems you contribute to and ensure new development aligns with strategic patterns.
• Stay current with AI industry trends/tooling and best practices including breakthroughs in LLMs, agents, and model safety.
• Participate in regular attendance and complete mandatory training programs as assigned by Acuity.
• Perform other duties as assigned.
Qualifications:
Required:
• Bachelor’s degree or equivalent combination of education and experience in software engineering, computer science, or related field.
• 5 or more years of experience in software engineering, including experience integrating AI capabilities into production applications using external services or APIs.
• Expertise across a wide range of tools and technologies, including programming languages such as Java, Python, and JavaScript.
• Experience with relational and non-relational databases including Db2, PostgreSQL, SQL Server, and vector databases.
• Familiarity with web UI frameworks such as Vue, React, or Angular.
• Knowledge of software development workflows, CI/CD pipelines and version control tools.
• Experience with AI and Gen AI technologies including LLMs, Retrieval-Augmented Generation (RAG), standardized model connection protocols, agent frameworks, and prompt engineering.
• Experience with AI Orchestration frameworks, like LangChain, to connect LLMs to data.
• Familiarity with AI Observability tools to monitor and debug AI responses.
• Understanding API Management, including rate limiting, latency, and token cost optimization.
• Experience with cloud platforms including Azure, IBM, GCP, or AWS for AI services.
• Ability to analyze user requirements, define technical specifications, estimate work, and deliver high quality solutions.
• Strong analytical thinking with the ability to break down complex problems and apply practical AI driven approaches.
• Strong communication skills and ability to work with cross functional teams.
• Excellence in documentation, testing practices, and operational reliability.
• Ability to stay current with AI advancements, LLM research, and evolving best practices.
• Passion for building reliable, safe, and impactful Generative AI systems.
Company:
Acuity Insurance is a leading provider of insurance solutions, delivering exceptional coverage and customer service to individuals and businesses in over 30 states. Founded in 1925, the company is headquartered in Sheboygan, Wisconsin, US, , with a team of 1001-5000 employees. The company is currently Late Stage.