ZipRecruiter

Log In

1

Ai Test Engineer Jobs in Alabama (NOW HIRING)

Ignite Now

TEST ENGINEER-JOURNEYMAN

Huntsville, AL · On-site

Job Title TEST ENGINEER-JOURNEYMAN Location Location Category Category Job Type Full-Time ... Experience supporting AI/ML-enabled autonomous systems. * Knowledge of systems engineering ...

IGNITE

TEST ENGINEER-JOURNEYMAN

Huntsville, AL · On-site

Job Title TEST ENGINEER-JOURNEYMAN Location Huntsville, AL 35806 US (Primary) Category Engineering ... AI/ML, space and missile defense intelligence, EMSO, advanced analytics, and programmatic domains.

Transform9

Automation Test Engineer

Birmingham, AL · On-site

$43 - $57/hr

Transform9 builds AI agents that answer every call for physician practices -- scheduling ... As a senior automation testing engineer, you'll own the test strategy and automation infrastructure ...

Transform9

Automation Test Engineer

Birmingham, AL · On-site +1

$43 - $57/hr

Transform9 builds AI agents that answer every call for physician practices - scheduling ... As a senior automation testing engineer, you'll own the test strategy and automation infrastructure ...

Transform9

Automation Test Engineer

Birmingham, AL · On-site

$43 - $57/hr

Transform9 builds AI agents that answer every call for physician practices - scheduling ... As a senior automation testing engineer, you'll own the test strategy and automation infrastructure ...

Via Logic LLC

Production Test Engineer

Huntsville, AL · On-site

$59K - $106K/yr

... AI&T. Basic Qualifications * BS degree in mechanical engineering, aerospace engineering, or ... Organizational skills to document test procedures, test reports and support configuration ...

Leidos

Production Test Engineer

Huntsville, AL · On-site

$59K - $106K/yr

... AI&T. Basic Qualifications * BS degree in mechanical engineering, aerospace engineering, or ... Organizational skills to document test procedures, test reports and support configuration ...

Leidos

Production Test Engineer

Huntsville, AL · On-site

$59K - $106K/yr

... AI&T. Basic Qualifications * BS degree in mechanical engineering, aerospace engineering, or ... Organizational skills to document test procedures, test reports and support configuration ...

next page

Showing results 1-20

All JobsAi Test Engineer JobsAi Test Engineer Jobs in Alabama

Ai Test Engineer information

See Alabama salary details

$16

$40

$67

How much do ai test engineer jobs pay per hour?

As of Sep 11, 2026, the average hourly pay for ai test engineer in Alabama is $40.10, according to ZipRecruiter salary data. Most workers in this role earn between $30.29 and $47.50 per hour, depending on experience, location, and employer.

What is an AI test engineer?

AI Test Engineers are professionals who design, develop, and execute tests to evaluate the performance, accuracy, and reliability of artificial intelligence systems and machine learning models. They work closely with data scientists and software developers to ensure AI solutions function as intended, identifying bugs, biases, and potential risks in algorithms. Their responsibilities often include creating test cases, automating test processes, and analyzing results to improve AI system quality and compliance.

What are the key skills and qualifications needed to thrive as an AI test engineer?

To thrive as an AI Test Engineer, you need a strong background in computer science, software testing methodologies, and a good understanding of machine learning concepts, often supported by a relevant degree. Familiarity with programming languages like Python, testing frameworks such as pytest, and tools like TensorFlow or PyTorch, along with certifications in software testing or AI, are typically required. Analytical thinking, attention to detail, and effective communication set top performers apart in this role. These skills and qualities are crucial for ensuring the reliability, accuracy, and ethical deployment of AI systems.

What are some common challenges AI test engineers face when validating machine learning models?

AI Test Engineers often encounter challenges such as ensuring the quality and fairness of machine learning models, identifying edge cases that the model may not handle well, and working with limited labeled data for testing. Additionally, interpreting test results can be complex due to the probabilistic nature of AI outputs, requiring close collaboration with data scientists to understand model behaviors. Effective communication and a strong foundation in both software testing and AI concepts are essential to address these challenges and ensure reliable AI solutions.

What is the difference between Ai Test Engineer vs Data Scientist?

AspectAi Test EngineerData Scientist
Required CredentialsBachelor's in CS, AI, or related field; knowledge of testing toolsBachelor's or higher in CS, Statistics, or related; proficiency in data analysis
Work EnvironmentSoftware testing teams, AI development projectsData analysis teams, AI research projects
Employer & Industry UsageTech companies, AI startups, software firmsTech companies, finance, healthcare, research institutions
Common Search & ComparisonOften compared for roles in AI testing and quality assuranceCompared for data analysis and AI model development roles

While both roles work within AI and tech environments, Ai Test Engineers focus on testing and validating AI systems, ensuring quality and performance. Data Scientists analyze data to develop models and insights. The roles are complementary but distinct in their core responsibilities.

How do I become an AI Test Engineer?

To become an AI Test Engineer, candidates typically need a strong background in computer science, software testing, or related fields, along with knowledge of AI and machine learning concepts. Skills in programming languages such as Python or Java, experience with testing tools, and understanding of AI model evaluation are essential. Earning relevant certifications and gaining experience in software development and testing environments can also improve job prospects.

What are popular job titles related to Ai Test Engineer jobs in Alabama?

For Ai Test Engineer jobs in Alabama, the most frequently searched job titles are:

What job categories do people searching Ai Test Engineer jobs in Alabama look for?

The top searched job categories for Ai Test Engineer jobs in Alabama are:

What cities in Alabama are hiring for Ai Test Engineer jobs?

Cities in Alabama with the most Ai Test Engineer job openings:

Ai Test Engineer jobs near you
Infographic showing various Ai Test Engineer job openings in Alabama as of August 2026, with employment types broken down into 1% As Needed, 82% Full Time, 11% Part Time, 5% Contract, and 1% Nights. Highlights an 93% Physical, 1% Hybrid, and 6% Remote job distribution, with an average salary of $83,412 per year, or $40.1 per hour.

Senior AI Test Automation Engineer

Birmingham, AL • On-site

Genuine Parts Company
Retail • 10K+ employees
Learn more about Genuine Parts Companyexternal

Full-time

Posted 9 days ago

Genuine Parts Company rating

7.3

Company rating: 7.3 out of 10

Based on 63 frontline employees who took The Breakroom Quiz

Job description

Senior AI Test Automation Engineer
Summary
The Senior AI Test Automation Engineer designs, develops, maintains, and executes automated testing and evaluation solutions for both traditional software and LLM-powered applications. Operating in a forward-deployed capacity, this role works directly with users and delivery teams to capture real-world usage patterns and feedback, translating them into a continuously growing evaluation framework that validates LLM behavior against the consistent flows users actually follow. This role partners with delivery teams, QA, development, product, AI engineering, and the QA Center of Excellence (CoE) to establish scalable automation and evaluation practices, increase test and eval coverage, and integrate quality controls throughout the software delivery lifecycle , from pre-deployment regression testing through production observability.
You must be eligible to work in the US without Visa Sponsorship.
Responsibilities
LLM Evaluation & Observability (Core)
  • Act as a forward-deployed quality engineer: engage directly with users and stakeholders to collect feedback, observe real usage patterns, and identify the consistent flows users follow through LLM-powered features.

  • Translate user feedback and production traces into curated evaluation datasets in LangSmith, and continuously expand the eval framework as new feedback, edge cases, and failure modes are discovered.

  • Design, build, and maintain offline evaluation suites (regression, benchmarking, and backtesting) that gate prompt, model, and LangGraph workflow changes before deployment.

  • Develop and calibrate evaluators , heuristic/code-based checks, LLM-as-judge evaluators, and pairwise comparisons , and validate judge reliability against human review.

  • Instrument and maintain end-to-end tracing across LangGraph agents and workflows using LangSmith, ensuring trace coverage, quality, and useful metadata for debugging and analysis.

  • Manage annotation queues and human-in-the-loop feedback workflows, routing interesting or problematic production runs to reviewers and feeding results back into datasets and evaluator calibration.

  • Analyze agent trajectories and multi-step LangGraph executions (tool calls, state transitions, retrieval steps) to pinpoint failure points and distinguish nondeterministic LLM variance from genuine product defects.

  • Integrate eval runs into CI/CD pipelines so that dataset versions, experiments, and quality thresholds provide automated feedback on every relevant change.

  • Support the adoption of production auditing and monitoring capabilities - such as online evaluations on live traffic, quality drift detection, and alerting - to help teams detect issues in production (supportive to the role, not its core focus).

Test Automation (Core)
  • Design, develop, and implement automated test scripts for UI, API, integration, and regression testing, including deterministic E2E coverage of LLM-powered application surfaces.

  • Integrate automated tests into CI/CD pipelines to enable timely feedback and continuous quality validation.

  • Collaborate with QA, development, product, and business teams to translate requirements, acceptance criteria, and expected agent behaviors into effective automated test and eval coverage.

  • Analyze and triage automation test failures, differentiating framework or script issues from valid product defects , including the added dimension of expected LLM nondeterminism.

  • Support test data management (including eval dataset versioning, splits, and provenance) and help identify or resolve test-environment stability issues.

  • Report on test execution results, eval experiment outcomes, automation and eval coverage, quality trends, and risks.

  • Participate in code reviews and contribute to automation and evaluation standards, reusable components, and best practices.

  • Engage with the QA CoE to align automation and AI evaluation practices with enterprise standards while contributing domain-specific feedback, lessons learned, and continuous-improvement opportunities.

Required Experience
  • 5+ years of experience in test automation engineering, software quality assurance, or a related role.

  • 1-2+ years of hands-on experience testing or evaluating LLM-powered applications, including building eval datasets, defining pass/fail criteria for nondeterministic outputs, and using LLM-as-judge or heuristic evaluators.

  • Hands-on experience with LangSmith for tracing, evaluation, and observability of LLM applications , including creating datasets, running experiments, and configuring evaluators.

  • Hands-on experience with LangGraph, including graph-based agent workflows, state and context management, and tool calling.

  • Hands-on experience building and maintaining automated test suites using Playwright, preferably with TypeScript.

  • Working proficiency in Python and/or TypeScript sufficient to author custom evaluators, tracing instrumentation, and test code.

  • Experience automating UI and API testing, and experience with API testing tools or frameworks.

  • Familiarity with CI/CD tools such as Azure DevOps, Jenkins, GitHub Actions, or equivalent, including integrating eval runs as pipeline gates.

  • Working knowledge of source-code version control, including Git.

  • Understanding of Agile/Scrum delivery practices and participation in Agile ceremonies.

  • Ability to analyze requirements and user feedback, identify test and eval scenarios, and create maintainable automated coverage.

  • Strong problem-solving, troubleshooting, communication, and collaboration skills , including comfort engaging directly with end users to gather feedback in a forward-deployed capacity.

Preferred / Nice to Have
  • Ability to understand and document end-to-end business processes, user journeys, and operational workflows.

  • Partner with business stakeholders and end users to translate process requirements into test scenarios, acceptance criteria, and LLM evaluation datasets.

  • Identify process exceptions, edge cases, dependencies, and risks that may affect application or agent behavior.

  • Validate that automated workflows and LLM-powered features produce outcomes aligned with defined business rules and user needs.

  • Use production feedback and observed user behavior to continuously refine process coverage, test automation, and evaluation frameworks.

  • Experience with online evaluations, production monitoring, and quality drift detection for LLM applications.

  • Experience with agent trajectory evaluation, RAG evaluation (retrieval relevance, groundedness, hallucination detection), or guardrails validation.

  • Familiarity with prompt engineering and prompt versioning workflows, and evaluating the impact of prompt or model changes.

  • Understanding of statistical approaches to nondeterministic testing (multiple-run sampling, confidence thresholds, summary metrics across datasets).

  • Experience authoring BDD/Gherkin scenarios using Cucumber or a similar framework.

  • Experience with contract testing tools.

  • Familiarity with SAFe and Agile Release Train (ART) practices.

  • Experience with AI-assisted engineering practices (e.g., using AI coding agents to accelerate test and eval development).

  • Experience with performance, accessibility, mobile, or security test automation.

  • Experience with test management and defect-tracking tools, such as Azure DevOps.

Not the right fit? Let us know you're interested in a future opportunity by joining our Talent Community on jobs.genpt.com or create an account to set up email alerts as new job postings become available that meet your interest!
GPC conducts its business without regard to sex, race, creed, color, religion, marital status, national origin, citizenship status, age, pregnancy, sexual orientation, gender identity or expression, genetic information, disability, military status, status as a veteran, or any other protected characteristic. GPC's policy is to recruit, hire, train, promote, assign, transfer and terminate employees based on their own ability, achievement, experience and conduct and other legitimate business reasons.

What Genuine Parts Company employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom

Apply