Test Environment Manager Jobs in Chicago, IL (NOW HIRING)

Overview:
At Zebra, we are a community of innovators who come together to create new ways of working. United by curiosity and a culture of caring, we develop smart solutions that anticipate our customer's and partner's needs and solve their challenges.
Being part of Zebra Nation means you are seen, heard, valued, and respected. Drawing from our unique perspectives, we collaborate to deliver on our purpose. Here you are part of a team pushing boundaries today to redefine the work of tomorrow for organizations, their employees, and those they serve.
You'll have opportunities to learn and lead in a forward-thinking environment, defining your path to a fulfilling career while channeling your skills toward causes you care about-locally and globally.
Come make an impact every day at Zebra.
What We're Looking For:
The AI Quality Analyst is a critical role responsible for ensuring the performance, safety, and reliability of our cutting-edge AI/ML models. You will be at the forefront of our development lifecycle, designing and executing comprehensive evaluation strategies to identify model weaknesses, potential biases, and critical edge cases. This role requires a blend of analytical rigor, technical aptitude, and a deep curiosity for how AI models behave in real-world scenarios. You will not just find bugs, but provide the actionable insights that drive model improvement and guide our research and development efforts.Responsibilities

• Evaluation Strategy & Benchmark Development: Design, develop, and maintain a comprehensive suite of test cases and evaluation benchmarks. Proactively identify potential model failure points, including edge cases, adversarial inputs, and sources of bias.

• Error Analysis & Failure Triage: Conduct systematic error analysis to categorize model failures and identify underlying patterns. Triage defects, prioritize them based on severity and impact, and work with the development team to ensure resolution.

• Data Sourcing & Curation: Source, curate, and manage high-quality datasets for model evaluation and testing. This includes performing data annotation and validation to ensure the integrity of our ground-truth data.

• Exploratory & Adversarial Testing (Red Teaming):Perform unscripted, exploratory testing to discover unexpected model behaviors. Participate in red teaming exercises to intentionally challenge our models and identify potential safety and security vulnerabilities.

• Test Environment Management: Set up, maintain, and troubleshoot testing and demonstration environments to ensure a stable and reliable evaluation pipeline.

• Reporting & Insights: Analyze and synthesize test results into clear, actionable reports for both technical and non-technical stakeholders. Translate complex findings into concrete recommendations for model improvement.

• Process Improvement: Actively participate in post-hoc evaluation reviews and contribute to the continuous improvement of our testing methodologies, tools, and overall quality assurance processes.

Qualifications and Skills

• Proven experience in a quality assurance, testing, or data analysis role, preferably within the AI/ML domain.
• A deep understanding of the machine learning lifecycle and the common failure modes of AI models.
• Hands-on experience with data annotation, data validation, and managing large datasets.
• Meticulous attention to detail and a methodical approach to problem-solving.
• Strong analytical skills with the ability to identify patterns in data and draw meaningful conclusions.
• Expertise with industry-standard test automation tools and libraries (e.g., Selenium, Playwright, Cypress, REST-assured).
• Experience in testing across different platforms (e.g. comprehensive testing of mobile Android/iOS and web applications)
• Experience with bug tracking systems (e.g., Jira) and test case management tools.
• Scripting skills (e.g., Python) for test automation and data manipulation.
• (Preferred) experience in testing AI systems, including evaluating agentic responses, model performance metrics, and data integrity.
• Excellent communication skills, with the ability to clearly document bugs and articulate complex technical issues.
• Familiarity with computer vision or other specific AI domains relevant to our work.

Equal Opportunity Employer:
Zebra is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability and protected veteran status, or any other basis prohibited by law. If you are an individual with a disability and need assistance in applying for a position, please contact us at workplace.accommodations@zebra.com.
Know Your Rights:
https://www.eeoc.gov/sites/default/files/2022-10/EEOC_KnowYourRights_screen_reader_10_20.pdf
Conozca sus Derechos:
https://www.eeoc.gov/sites/default/files/2022-10/22-088_EEOC_KnowYourRightsSp_10_20.pdf
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation at workplace.accommodations@zebra.com .
Pay Range:
$122,800.00 - $184,200.00 Annual
Incentive Compensation:
In addition to base pay, Zebra offers this role the opportunity to earn a performance-based annual cash incentive, at a target equal to 12% of base pay, in accordance with the terms of the applicable incentive plan.
Zebra Total Rewards:
Zebra Total Rewards includes more than just pay and is structured to meet the needs of our changing global business and evolving talent. We are committed to providing our employees with a benefits program that is comprehensive and competitive - including healthcare, wellness, inclusion networks, and continued learning and development offerings. We offer community service days, in addition to the traditional insurances, compensation, parental leave, employee assistance program and paid time off offerings depending on the country where you work.
Salary offered will vary depending on your location, job-related skills, knowledge, and experience.
Additionally, all Zebra roles are eligible for cash incentive programs. For example, sales roles have additional opportunity to earn substantial variable compensation tied to quota achievement. In most other roles, the Zebra annual cash incentive program links Company and individual performance together. Some roles may also be eligible for long-term incentive equity awards.
Benefits:
We understand the importance of work-life balance and wellbeing, which is why we offer flexibility for our teams including: hybrid work, adaptable hours, Summer Flex Fridays, Focus Fridays, and an annual companywide well-being day to promote revitalization and success.
Job Posting Statement:
To protect candidates from falling victim to online fraudulent activity involving fake job postings and employment offers, please be aware our recruiters will always connect with you via @zebra.com email accounts. Applications are only accepted through our applicant tracking system and only accept personal identifying information through that system. Our Talent Acquisition team will not ask for you to provide personal identifying information via e-mail or outside of the system. If you are a victim of identity theft contact your local police department.
AI Technology Statement:
Zebra Technologies leverages AI technology to evaluate job applications using objective, job-relevant criteria. This approach enhances efficiency and promotes fairness in the hiring process. However, every decision regarding interviews and hiring is made by our dedicated team, because we believe people make the best decisions about people. For more on how we use technology in hiring and how we process applicant data, see our Zebra Privacy Policy.