Python Automation Developer Jobs in Oak Brook, IL

Job Title & LocationAI Automation Engineer - VirtualAlight Solutions – Lincolnshire, ILFull-timeMedical, Life, Retirement, PTOPosted 5 hours agoJob DescriptionAs the AI Automation Engineer in the AICoE, you will be instrumental in standing up the initial automation framework for AI Quality. You'll work closely with the AI Quality Lead and developers to build, run, and scale automated test suites for AI models and agents.ResponsibilitiesDevelop and maintain automated test frameworks for LLM, RAG, agentic, and classic ML systemsImplement golden/adversarial test cases (functional, safety, bias, prompt injection, refusal correctness, tool use)Integrate evaluation tools (promptfoo, Ragas) into CI/CD pipelines and support batch quality assessmentsAuthor and validate OpenAPI/JSON Schema contracts for model inputs/outputs; enforce schema checks in pipelinesInstrument observability metrics (latency, success rate, cost, drift) using OpenTelemetry and export to dashboardsSupport data QA, lineage tracking, and drift monitoring (Evidently, OpenLineage)Collaborate with AI developers, SDETs, and SRE to troubleshoot, optimize, and document automation processesContribute to red-team playbooks and security test books mapped to OWASP/ATLAS risksRequirementsExperience in test automation, software engineering, or quality assurance (AI/ML experience is a plus, but not required)Proficiency in Python, JavaScript/Node.js, or similar programming languagesFamiliarity with CI/CD, version control, and cloud-based deployment environmentsExperience with test automation tools (promptfoo, Ragas, Postman)Understanding of AI model evaluation metrics (faithfulness, answer relevancy, hallucination, refusal correctness)Basic knowledge of fairness, safety, and security testing for AI systemsExperience with AI infrastructure, orchestration, data integrity, deployments, and model registry practicesInterest in Responsible AI, bias mitigation, and regulatory complianceBachelor's degree in Computer Science, Engineering, or related field, or equivalent practical experienceBenefitsWe offer programs and plans for a healthy mind, body, wallet, and life because it's important our benefits care for the whole person. Options include a variety of health coverage options, wellbeing and support programs, retirement, vacation and sick leave, maternity, paternity & adoption leave, continuing education and training as well as several voluntary benefit options.Equal Opportunity Policy StatementAlight is an Equal Employment Opportunity employer and does not discriminate against anyone based on sex, race, color, religion, creed, national origin, ancestry, age, physical or mental disability, medical condition, pregnancy, marital or domestic partner status, citizenship, military or veteran status, sexual orientation, gender, gender identity or expression, genetic information, or any other legally protected characteristics or conduct covered by federal, state, or local law.

In addition, we take affirmative action to employ disabled persons, disabled veterans, and other covered veterans. Alight provides reasonable accommodations to the known limitations of otherwise qualified employees and applicants for employment with disabilities and sincerely held religious beliefs, practices and observances unless doing so would result in undue hardship. Applicants for employment may request a reasonable accommodation/modification by contacting their recruiter.Background ChecksBy applying for a position with Alight, you understand that, should you be made an offer, it will be contingent on your undergoing and successfully completing a background check consistent with Alight's employment policies.

Background checks may include SSN/SIN validation, education verification, employment verification, and criminal check, search against global sanctions and government watch lists, credit check, and/or drug test. You will be notified during the hiring process which checks are required by the position.J-18808-Ljbffr