Machine Learning Qa Engineer Jobs (NOW HIRING)

Job Description

The Quality Assurance (QA) Engineer plays a pivotal role in ensuring the quality, reliability, and performance of our web applications and AI-driven solutions. This role is responsible for designing, building, and maintaining automated test frameworks, conducting hands-on exploratory and UI/API testing, and validating non-deterministic outputs from machine learning models.

In addition to traditional QA practices, you will leverage modern containerization tools (such as Docker) to manage isolated test environments and utilize AI-assisted tools (such as Copilot, synthetic test data generators, and intelligent test execution platforms) to optimize testing workflows, improve test coverage, and accelerate release cycles.

Duties and Responsibilities

• Modern Web & API Automation:

• Design, develop, and maintain robust, scalable automation frameworks for modern single-page web applications (React, Angular, Vue) and microservices/REST APIs.
• Integrate automated test suites into CI/CD deployment pipelines (e.g., GitHub Actions, Azure DevOps, Jenkins) for continuous validation.

• Containerized Test Environments & Infrastructure:

• Spin up, manage, and tear down local and cloud-based test environments using Docker containers and container orchestration tools.
• Execute automated test suites inside containerized environments to ensure consistency across local, staging, and production-like setups.

• AI & Model Output Testing:

• Develop test strategies to validate AI/ML features, large language model (LLM) outputs, and non-deterministic application behavior for accuracy, relevancy, latency, and edge cases.
• Monitor and measure AI output consistency, safety/bias guardrails, and system performance under varying prompt payloads.

• AI-Driven QA Optimization:

• Utilize AI-assisted development tools and intelligent QA platforms to generate test cases, synthesize realistic test data sets, and accelerate root-cause failure analysis.

• Cross-Functional Collaboration:

• Collaborate closely with Product Owners, Software Engineers, and Data/ML Engineers during Agile/Scrum sprints to refine requirements, acceptance criteria, and edge-case scenarios.
• Participate in code reviews for test automation scripts to ensure high code quality and maintainability.

• Quality Process & Defect Management:

• Identify, document, prioritize, and track defects using modern issue-tracking tools (e.g., Jira).
• Perform exploratory, regression, performance, and cross-browser/device testing across staging and production environments.
• Assist in setting QA standards, test metrics, and best practices across the engineering organization.

Qualifications

Core Technical Requirements

• Automation Frameworks: Hands-on experience with modern web automation and end-to-end testing tools (e.g., Playwright, Cypress, Selenium, or WebdriverIO).
• Containerization & DevOps: Solid practical knowledge of Docker (writing Dockerfiles, using Docker Compose) for spinning up containerized applications, databases, and automated testing nodes.
• API Testing: Proficiency in API testing and automation using tools like Postman, REST Assured, or HTTP clients.
• Modern Web Technologies: Solid understanding of modern web architectures (HTML5, CSS3, JavaScript/TypeScript, DOM manipulation, asynchronous state management).
• Programming & Scripting: Strong coding skills in at least one modern language (e.g., TypeScript/JavaScript, Python, C#, or Java).
• Database Querying: Proficiency with SQL (PostgreSQL, SQL Server, MySQL) and experience querying relational or document databases for test setup and verification.
  

AI & Emerging Tech Capabilities

• AI Feature Testing: Familiarity with concepts around evaluating AI/ML features, RAG architectures, prompt engineering testing, or heuristic evaluation metrics.
• AI Productivity Tools: Familiarity or experience using AI tools (e.g., GitHub Copilot, ChatGPT, Claude) to write test scripts, create mock data, or generate test scenarios
  

Professional & Interpersonal Skills

• Agile Methodology: Proven experience working in an Agile/Scrum environment.
• Problem-Solving: Exceptional analytical, critical thinking, and debugging skills with high attention to detail.
• Communication: Excellent written and verbal communication skills to articulate technical defects, risk assessments, and test coverage clearly to technical and non-technical stakeholders.

Education/Experience

A bachelor's degree is not required, but preferred. More than anything, RMS is looking for people who are passionate about technology who want to develop the skills to solve hard and important problems.

Environmental Conditions

Indoor climate-controlled environment. Moderate to quiet noise level

Physical requirements

While performing the duties of this Job, the employee is regularly required to communicate verbally and in the written form. The employee is physically required to utilize a laptop and other electronic devices effectively. The employee must lift and/or move up to 20 pounds (laptop computer, bag, and accessories). Specific vision abilities required by this job include close vision and distance vision.

All applicants are subject to drug screens and background checks per company policies.