ZipRecruiter

Log In

1

Generative Ai Testing Jobs (NOW HIRING)

Cotiviti

$32 - $40/hr

Conduct research on Agentic AI models and their potential impact on healthcare workflows and decision-making. * Assist in designing, developing, and testing autonomous AI agents and generative AI ...

Jj

Data and Generative AI Engineer

New Brunswick, NJ · On-site

$118K - $141K/yr

Design, develop and deliver Generative/Agentic AI solutions that accelerate End-to-End Data ... Ensure data quality and integrity through meticulous testing and validation, applying Intelligent ...

Cotiviti

Intern - Generative AI/Agentic AI/Research

$32 - $40/hr

Conduct research on Agentic AI models and their potential impact on healthcare workflows and decision-making. * Assist in designing, developing, and testing autonomous AI agents and generative AI ...

Jj

Data and Generative AI Engineer

Raritan, NJ · On-site

$117K - $140K/yr

Design, develop and deliver Generative/Agentic AI solutions that accelerate End-to-End Data ... Ensure data quality and integrity through meticulous testing and validation, applying Intelligent ...

VTG

AI Architect

Chantilly, VA · On-site

Experience with AI testing, validation, benchmarking, and evaluation frameworks for both traditional ML and generative AI systems. * Experience implementing practical MLOps pipelines and AI ...

SelectMinds

AI Architect Agentic & Generative AI

Dallas, TX · On-site

$180K - $200K/yr

... testing. * Collaborate with platform and DevOps teams to enable governed AWS landing zones, CI/CD ... generative AI solutions in production environments. * Hands-on experience with Amazon Bedrock ...

next page

Showing results 1-20

All JobsGenerative Ai Testing Jobs

Generative Ai Testing information

See salary details

$31

$53

$76

How much do generative ai testing jobs pay per hour?

As of Jun 29, 2026, the average hourly pay for generative ai testing in the United States is $53.73, according to ZipRecruiter salary data. Most workers in this role earn between $44.23 and $61.54 per hour, depending on experience, location, and employer.

What is the difference between Generative Ai Testing vs Data Scientist?

AspectGenerative Ai TestingData Scientist
Required CredentialsKnowledge of AI models, testing tools, programming skillsStatistics, programming, data analysis certifications
Work EnvironmentAI development teams, testing labs, tech companiesResearch labs, tech firms, finance, healthcare
Employer & Industry UsageAI product testing, quality assurance in techData analysis, predictive modeling across industries

Generative Ai Testing focuses on evaluating and validating AI-generated content and models, ensuring quality and accuracy. Data Scientists analyze data, build models, and derive insights. While both roles require programming and AI knowledge, Generative Ai Testing emphasizes testing processes, whereas Data Scientists focus on data analysis and model development.

How much do AI testers get paid?

AI testers, involved in evaluating and validating generative AI models, typically earn salaries ranging from $60,000 to $120,000 annually depending on experience, location, and company size. Entry-level positions may start lower, while experienced testers with specialized skills in machine learning and data analysis can earn higher wages.

Is AI testing a good career?

AI testing, including roles like Generative AI Testing, is a growing field with increasing demand for skills in machine learning, data analysis, and programming. It offers opportunities in tech companies, research labs, and startups, often requiring knowledge of AI frameworks and testing tools. The career can be stable and rewarding for those with technical expertise and an interest in AI development and quality assurance.

What are the key skills and qualifications needed to thrive as a Generative AI Testing Specialist, and why are they important?

To thrive as a Generative AI Testing Specialist, you need a robust understanding of machine learning principles, model evaluation techniques, and a background in computer science or a related field. Familiarity with tools such as Python, TensorFlow, PyTorch, and model evaluation frameworks, as well as experience with automated testing platforms, is typically required. Analytical thinking, attention to detail, and strong communication skills help you identify model weaknesses and collaborate effectively with development teams. These skills are crucial to ensure the reliability, safety, and ethical deployment of generative AI solutions.

What are some common challenges faced when testing generative AI models, and how can I prepare to address them in this role?

Testing generative AI models often involves unique challenges such as evaluating the quality and relevance of generated content, detecting bias or inappropriate outputs, and ensuring model consistency across various prompts. You may work closely with data scientists and engineers to create robust evaluation frameworks and develop automated as well as manual testing strategies. Familiarity with prompt engineering, statistical evaluation techniques, and domain-specific knowledge will help you address these challenges effectively. Proactively staying updated on industry best practices and collaborating with cross-functional teams are key to success in this dynamic field.

What is a $900,000 AI job?

A $900,000 AI job typically refers to high-level roles in artificial intelligence, such as senior machine learning engineers, AI research directors, or executive positions, often requiring advanced skills, extensive experience, and sometimes specialized certifications. These roles usually involve leading AI development projects, strategic planning, and overseeing AI teams in large organizations or tech companies.

How do I become an AI tester?

To become an AI tester, you should have a strong understanding of machine learning concepts, programming skills in languages like Python, and experience with data annotation and model evaluation. Familiarity with AI development tools and testing frameworks, along with attention to detail, is essential for identifying issues in AI systems.

What is Generative AI Testing?

Generative AI Testing refers to the process of evaluating and validating AI systems, particularly those that generate content such as text, images, or code. This type of testing focuses on assessing the accuracy, reliability, fairness, and safety of generative models to ensure they function as intended and avoid producing harmful or biased outputs. Testers use various methods, including automated and manual techniques, to check for issues like hallucinations, inappropriate content, or security vulnerabilities. The goal is to build trust in generative AI systems and ensure they meet quality and ethical standards before deployment.
More about Generative Ai Testing jobs
What cities are hiring for Generative Ai Testing jobs? Cities with the most Generative Ai Testing job openings:
What states have the most Generative Ai Testing jobs? States with the most job openings for Generative Ai Testing jobs include:
What job categories do people searching Generative Ai Testing jobs look for? The top searched job categories for Generative Ai Testing jobs are:
Generative Ai Testing jobs near you
Infographic showing various Generative Ai Testing job openings in the United States as of June 2026, with employment types broken down into 97% Full Time, and 3% Contract. Highlights an 66% Physical, 3% Hybrid, and 31% Remote job distribution, with an average salary of $111,750 per year, or $53.7 per hour.
Senior MLOps & Generative AI Engineer - Remote

Senior MLOps & Generative AI Engineer - Remote

Sentara Healthcare

Centre, AL • On-site, Remote

$98K - $135K/yr

Full-time

Medical, Dental, Vision, Life, Retirement, PTO

Posted 14 hours ago

Sentara Health rating

6.8

Company rating: 6.8 out of 10

Based on 385 frontline employees who took The Breakroom Quiz

483rd of 877 rated healthcare providers

Job description

City/State
Virginia Beach, VA
Work Shift
First (Days)
Overview:
Sentara is hiring a Senior MLOps & Generative AI Engineer!
This position is fully remote!
Candidates must reside in one of the following states:
Alabama, Delaware, Florida, Georgia, Idaho, Indiana, Kansas, Louisiana, Maine, Maryland, Minnesota, Nebraska, Nevada, New Hampshire, North Dakota, Ohio, Oklahoma, Pennsylvania, South Carolina, South Dakota, Tennessee, Texas, Utah, Washington, West Virginia, Wisconsin, or Wyoming.
Overview
We are seeking a highly skilled and experienced Senior MLOps & Generative AI Engineer to join our growing AI organization and help advance current and future initiatives applying machine learning, deep learning, NLP, and Generative AI technologies to improve healthcare outcomes and operational excellence.
This role combines two critical focus areas:
  • MLOps Engineering - building and scaling enterprise-grade ML infrastructure, deployment pipelines, observability, governance, and automation capabilities.
  • Generative AI Engineering - designing, architecting, deploying, and optimizing secure, production-ready GenAI applications and platforms leveraging LLMs, RAG architectures, vector databases, prompt orchestration, and AI evaluation frameworks.

As a Senior Engineer, you will partner closely with AI Scientists, Data Engineers, Software Engineers, Architects, and Product teams to operationalize AI/ML and Generative AI solutions at enterprise scale. You will play a key role in shaping the organization's AI platform strategy, driving best practices, and delivering scalable, secure, and reliable AI systems in production healthcare environments.
Key Responsibilities
MLOps Engineering Responsibilities
  • Design, build, and maintain scalable ML infrastructure and pipelines supporting model training, deployment, monitoring, governance, and lifecycle management.
  • Develop and optimize CI/CD pipelines for machine learning and AI workloads across development, staging, and production environments.
  • Build reusable ML platform capabilities including feature stores, model registries, experimentation frameworks, artifact management, and deployment automation.
  • Implement scalable orchestration and workflow solutions for batch and real-time ML inference workloads.
  • Create robust monitoring systems to measure model performance, detect model drift, monitor data quality, and ensure production reliability.
  • Develop automation tools and self-service capabilities to improve the efficiency, scalability, and reliability of MLOps processes.
  • Collaborate with Data Scientists and Software Engineers to streamline the ML lifecycle from experimentation through enterprise production deployment.
  • Apply software engineering best practices to AI/ML systems including testing, observability, resiliency, security, versioning, and infrastructure-as-code.
  • Identify gaps and improvement opportunities within the organization's ML platform ecosystem and architect scalable solutions to address them.
  • Support enterprise AI governance, compliance, auditability, and model risk management requirements.
  • Ensure platform scalability, reliability, security, and operational excellence across AI/ML systems.

Generative AI Engineering Responsibilities
  • Lead the architecture, design, and deployment of enterprise Generative AI solutions leveraging LLMs, foundation models, and agentic AI systems.
  • Design and implement Retrieval-Augmented Generation (RAG) pipelines using vector databases, embeddings, semantic search, reranking, and retrieval optimization strategies.
  • Build scalable LLM orchestration frameworks using technologies such as LangChain, LlamaIndex, Semantic Kernel, or equivalent frameworks.
  • Develop advanced prompt engineering strategies, prompt chaining, context management, and agent workflows to improve LLM accuracy and reliability.
  • Evaluate and implement fine-tuning, parameter-efficient tuning, and prompt-based optimization approaches for domain-specific use cases.
  • Build AI evaluation and benchmarking frameworks to measure hallucination rates, response quality, grounding accuracy, toxicity, bias, latency, and business performance metrics.
  • Implement AI safety guardrails, governance controls, content filtering, and responsible AI practices for enterprise healthcare environments.
  • Design scalable GenAI APIs and microservices supporting high-throughput enterprise AI applications.
  • Optimize GenAI systems for cost, latency, throughput, and inference performance across cloud and hybrid environments.
  • Integrate enterprise data sources, healthcare systems, and knowledge repositories into secure GenAI workflows.
  • Research and evaluate emerging GenAI technologies, open-source frameworks, and foundation models to drive innovation and continuous improvement.
  • Develop architecture diagrams, technical roadmaps, implementation strategies, and executive-level documentation for enterprise AI initiatives.
  • Collaborate with cybersecurity, compliance, and infrastructure teams to ensure secure and compliant deployment of GenAI solutions involving PHI and sensitive healthcare data.
  • Contribute to the development of AI platform standards, reusable GenAI accelerators, templates, and engineering best practices.

Required Qualifications
  • 5+ years of experience building and deploying production software, ML systems, or AI platforms.
  • 1+ years of hands-on experience building production Generative AI or LLM-based applications.
  • Strong programming skills in Python and experience with software engineering best practices.
  • Experience with major deep learning and LLM frameworks such as PyTorch, Hugging Face Transformers, TensorFlow, or equivalent.
  • Hands-on experience implementing RAG architectures, vector search, embeddings, prompt engineering, and LLM orchestration frameworks.
  • Experience with vector databases such as Pinecone, Weaviate, Chroma, FAISS, Milvus, or equivalent technologies.
  • Experience deploying AI/ML systems in cloud environments including AWS, Azure, or GCP.
  • Strong understanding of APIs, distributed systems, microservices, and scalable backend architectures.
  • Experience with Kubernetes, containerization, orchestration, and cloud-native infrastructure.
  • Experience implementing CI/CD pipelines, infrastructure automation, and MLOps best practices.
  • Experience building monitoring, observability, and alerting solutions for ML and AI systems.
  • Strong understanding of AI/ML lifecycle management, governance, model versioning, and production operations.
  • Experience designing secure, scalable, production-ready AI platforms and services.
  • Strong communication and collaboration skills with the ability to work across technical and business teams.

Preferred Qualifications
  • Previous experience implementing Generative AI and MLOps solutions within healthcare environments.
  • Experience working with EPIC or healthcare interoperability platforms.
  • Understanding of HIPAA, PHI handling, healthcare compliance, and responsible AI practices.
  • Experience with AI governance frameworks, LLM evaluation methodologies, and AI safety tooling.
  • Experience with GPU infrastructure optimization and scalable inference architectures.
  • Familiarity with multi-agent AI systems and autonomous workflows.
  • Experience with event-driven architectures, streaming pipelines, and real-time inference systems.
  • Exposure to model fine-tuning techniques including LoRA, PEFT, RLHF, or domain adaptation strategies.
  • Experience with enterprise AI platform architecture and internal developer platforms.
  • Prior experience mentoring engineers and leading technical initiatives.

Education
  • 5+ years of relevant experience with a degree (Required)

or
  • 7+ years of relevant experience without a degree (Required)
  • Experience in lieu of Bachelor's Degree.

Certification/Licensure
  • No specific certification or licensure requirements

Experience
  • 5 to 7 years of relevant experience

We provide market-competitive compensation packages, inclusive of base pay, incentives, and benefits. The base pay rate for Full Time employment is: 91,416.00 - 152,380.80. Additional compensation may be available for this role such as shift differentials, standby/on-call, overtime, premiums, extra shift incentives, or bonus opportunities.
Keywords: Talroo-IT, MLOps, Gen AI, LLM, AWS, Azure, GCP, AI/ML, Python, PyTorch, Hugging Face Transformers, TensorFlow, RAG, EPIC, HIPAA, AI Governance
Benefits: Caring For Your Family and Your Career
• Medical, Dental, Vision plans
• Adoption, Fertility and Surrogacy Reimbursement up to 10,000
• Paid Time Off and Sick Leave
• Paid Parental & Family Caregiver Leave
• Emergency Backup Care
• Long-Term, Short-Term Disability, and Critical Illness plans
• Life Insurance
• 401k/403B with Employer Match
• Tuition Assistance - 5,250/year and discounted educational opportunities through Guild Education
• Student Debt Pay Down - 10,000
• Reimbursement for certifications and free access to complete CEUs and professional development
• Pet Insurance
• Legal Resources Plan
• Colleagues have the opportunity to earn an annual discretionary bonus if established system and employee eligibility criteria is met.
Sentara Health is an equal opportunity employer and prides itself on the diversity and inclusiveness of its close to an almost 30,000-member workforce. Diversity, inclusion, and belonging is a guiding principle of the organization to ensure its workforce reflects the communities it serves.
In support of our mission "to improve health every day," this is a tobacco-free environment.
For positions that are available as remote work, Sentara Health employs associates in the following states:
Alabama, Delaware, Florida, Georgia, Idaho, Indiana, Kansas, Louisiana, Maine, Maryland, Minnesota, Nebraska, Nevada, New Hampshire, North Carolina, North Dakota, Ohio, Oklahoma, Pennsylvania, South Carolina, South Dakota, Tennessee, Texas, Utah, Virginia, Washington, West Virginia, Wisconsin, and Wyoming.

What Sentara Health employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom

Apply