1

Generative Ai Testing Jobs in California (NOW HIRING)

Senior Applied AI Engineer

San Diego, CA · On-site

$110K - $152K/yr

Lead applied AI projects that integrate generative models into insurance products, customer ... Strong knowledge of MLOps practices, including CI/CD pipelines, model versioning, testing, and ...

This role focuses on creating advanced AI applications, including Generative AI systems like ... Ensure code quality and maintainability through unit testing, code reviews, and adherence to best ...

Senior Applied AI Engineer

San Diego, CA

$110K - $152K/yr

Lead applied AI projects that integrate generative models into insurance products, customer ... Strong knowledge of MLOps practices, including CI/CD pipelines, model versioning, testing, and ...

Senior Applied AI Engineer

San Diego, CA · On-site

$110K - $152K/yr

Lead applied AI projects that integrate generative models into insurance products, customer ... Strong knowledge of MLOps practices, including CI/CD pipelines, model versioning, testing, and ...

This role focuses on creating advanced AI applications, including Generative AI systems like ... Ensure code quality and maintainability through unit testing, code reviews, and adherence to best ...

AI Engineer with Java background

Sunnyvale, CA · On-site

$61.75 - $84.50/hr

Design, develop, and deploy AI-powered applications and solutions using modern LLMs and Generative ... Participate in architecture discussions, code reviews, testing, deployment, and ongoing support ...

Senior AI Engineer

Dublin, CA

$117K - $161K/yr

Build and productionize ML and Generative AI workflows , including prompt engineering, retrieval ... Implement prompt versioning, experimentation, and A/B testing to support rapid iteration and data ...

Senior AI Engineer

Dublin, CA

$117K - $161K/yr

Build and productionize ML and Generative AI workflows , including prompt engineering, retrieval ... Implement prompt versioning, experimentation, and A/B testing to support rapid iteration and data ...

Gen AI Architect

Santa Clara, CA · On-site +1

$74.50 - $98/hr

Architect enterprise-scale Generative AI solutions leveraging LLMs, embeddings, and retrieval ... Top 3 Must Haves: 1. Extensive Agentic AI, Gen AI app testing experience with dev background 2. ...

Senior AI Engineer

Pleasanton, CA · On-site

$116K - $159K/yr

Responsibilities : • Design, develop, and deploy machine learning and Generative AI solutions to ... testing • Practical experience deploying LLM applications with guardrails, prompt versioning ...

Knowledge of automated UI testing frameworks * Past experience or strong interest in generative AI * Highly collaborative mindset with strong communication and problem-solving skills. As a part of ...

Senior Software Engineer (AI/ML)

San Francisco, CA · On-site

$144K - $190K/yr

Enhance NLP clustering algorithms using Generative AI (LLMs, embeddings, semantic clustering ... Drive best practices in code quality, testing, and deployment * Stay current with emerging AI ...

next page

Showing results 1-20

Generative Ai Testing information

What is the difference between Generative Ai Testing vs Data Scientist?

AspectGenerative Ai TestingData Scientist
Required CredentialsKnowledge of AI models, testing tools, programming skillsStatistics, programming, data analysis certifications
Work EnvironmentAI development teams, testing labs, tech companiesResearch labs, tech firms, finance, healthcare
Employer & Industry UsageAI product testing, quality assurance in techData analysis, predictive modeling across industries

Generative Ai Testing focuses on evaluating and validating AI-generated content and models, ensuring quality and accuracy. Data Scientists analyze data, build models, and derive insights. While both roles require programming and AI knowledge, Generative Ai Testing emphasizes testing processes, whereas Data Scientists focus on data analysis and model development.

How much do AI testers get paid?

AI testers, involved in evaluating and validating generative AI models, typically earn salaries ranging from $60,000 to $120,000 annually depending on experience, location, and company size. Entry-level positions may start lower, while experienced testers with specialized skills in machine learning and data analysis can earn higher wages.

Is AI testing a good career?

AI testing, including roles like Generative AI Testing, is a growing field with increasing demand for skills in machine learning, data analysis, and programming. It offers opportunities in tech companies, research labs, and startups, often requiring knowledge of AI frameworks and testing tools. The career can be stable and rewarding for those with technical expertise and an interest in AI development and quality assurance.

What are the key skills and qualifications needed to thrive as a Generative AI Testing Specialist, and why are they important?

To thrive as a Generative AI Testing Specialist, you need a robust understanding of machine learning principles, model evaluation techniques, and a background in computer science or a related field. Familiarity with tools such as Python, TensorFlow, PyTorch, and model evaluation frameworks, as well as experience with automated testing platforms, is typically required. Analytical thinking, attention to detail, and strong communication skills help you identify model weaknesses and collaborate effectively with development teams. These skills are crucial to ensure the reliability, safety, and ethical deployment of generative AI solutions.

What are some common challenges faced when testing generative AI models, and how can I prepare to address them in this role?

Testing generative AI models often involves unique challenges such as evaluating the quality and relevance of generated content, detecting bias or inappropriate outputs, and ensuring model consistency across various prompts. You may work closely with data scientists and engineers to create robust evaluation frameworks and develop automated as well as manual testing strategies. Familiarity with prompt engineering, statistical evaluation techniques, and domain-specific knowledge will help you address these challenges effectively. Proactively staying updated on industry best practices and collaborating with cross-functional teams are key to success in this dynamic field.

What is a $900,000 AI job?

A $900,000 AI job typically refers to high-level roles in artificial intelligence, such as senior machine learning engineers, AI research directors, or executive positions, often requiring advanced skills, extensive experience, and sometimes specialized certifications. These roles usually involve leading AI development projects, strategic planning, and overseeing AI teams in large organizations or tech companies.

How do I become an AI tester?

To become an AI tester, you should have a strong understanding of machine learning concepts, programming skills in languages like Python, and experience with data annotation and model evaluation. Familiarity with AI development tools and testing frameworks, along with attention to detail, is essential for identifying issues in AI systems.

What is Generative AI Testing?

Generative AI Testing refers to the process of evaluating and validating AI systems, particularly those that generate content such as text, images, or code. This type of testing focuses on assessing the accuracy, reliability, fairness, and safety of generative models to ensure they function as intended and avoid producing harmful or biased outputs. Testers use various methods, including automated and manual techniques, to check for issues like hallucinations, inappropriate content, or security vulnerabilities. The goal is to build trust in generative AI systems and ensure they meet quality and ethical standards before deployment.
What are popular job titles related to Generative Ai Testing jobs in California? For Generative Ai Testing jobs in California, the most frequently searched job titles are:
What job categories do people searching Generative Ai Testing jobs in California look for? The top searched job categories for Generative Ai Testing jobs in California are:
What cities in California are hiring for Generative Ai Testing jobs? Cities in California with the most Generative Ai Testing job openings:
Infographic showing various Generative Ai Testing job openings in California as of June 2026, with employment types broken down into 13% Internship, 68% Full Time, 13% Part Time, and 6% Temporary. Highlights an 75% In-person, 6% Hybrid, and 19% Remote job distribution.

Senior Applied AI Engineer

Icwgroup

San Diego, CA • On-site

$110K - $152K/yr

Full-time

Medical, Dental, Vision, Retirement, PTO

Posted 20 days ago


Job description

Are you looking to make an impactful difference in your work, yourself, and your community? Why settle for just a job when you can land a career? At ICW Group, we are hiring team members who are ready to use their skills, curiosity, and drive to be part of our journey as we strive to transform the insurance carrier space. We're proud to be in business for over 50 years, and its change agents like yourself that will help us continue to deliver our mission to create the best insurance experience possible.

Headquartered in San Diego with regional offices located throughout the United States, ICW Group has been named for ten consecutive years as a Top 50 performing P&C organization offering the stability of a large, profitable and growing company combined with a focus on all things people. It's our team members who make us an employer of choice and the vibrant company we are today. We strive to make both our internal and external communities better everyday! Learn more about why you want to be here!

PURPOSE OF THE JOB

The Senior Applied AI Engineer is responsible for leading the design, development, and operationalization of generative AI solutions that support the insurance company's business objectives. This role drives the adoption of cutting-edge AI technologies, ensures models are deployed securely, cost-effectively, and in compliance with regulatory and organizational standards, and serves as a mentor and technical authority for the AI and ML Platform team. The engineer will work cross-functionally with cloud engineering, AI, and ML Ops teams to integrate AI capabilities into both customer-facing and internal systems, ensuring reliability, scalability, and measurable business impact.

ESSENTIAL DUTIES AND RESPONSIBILITIES

  • Design, implement, and maintain end-to-end generative AI solutions, including model fine-tuning, deployment, and monitoring in production environments.

  • Lead applied AI projects that integrate generative models into insurance products, customer engagement tools, and internal operational workflows.

  • Collaborate with cloud engineering, AI, and ML Ops engineers to operationalize AI workloads on AWS using services such as SageMaker, Lambda, ECS/EKS, and S3.

  • Build robust AI/ML pipelines leveraging Snowflake for feature engineering and data preprocessing.

  • Ensure AI/ML systems comply with organizational security policies, regulatory requirements, and data privacy standards.

  • Establish best practices for model governance, explainability, monitoring, and risk mitigation in regulated environments.

  • Monitor and optimize AI workloads for cost efficiency in cloud environments, balancing performance, scalability, and budget considerations.

  • Mentor mid-level engineers, reviewing code, architecture, and model design for quality and compliance.

  • Drive research and evaluation of emerging generative AI technologies and frameworks, recommending adoption strategies for the organization.

  • Troubleshoot performance and scalability issues in production AI systems and optimize models for cost and latency.

EDUCATION AND EXPERIENCE

  • Bachelor's or Master's degree in Computer Science, Data Science, Applied Mathematics, or a related technical discipline.

  • 5+ years of experience in AI/ML engineering, with at least 3 years focused on generative AI or large language models in applied settings.

  • Proven experience deploying and scaling AI/ML models in cloud environments, preferably AWS.

  • Hands-on experience with Snowflake for AI/ML feature engineering and data integration in a plus.

  • Prior experience in insurance, financial services, or highly regulated environments is strongly preferred.

CERTIFICATES, LICENSES, AND REGISTRATIONS

  • AWS Certified Machine Learning - Specialty or AWS Certified Solutions Architect preferred.

  • Optional: Relevant AI/ML certifications (e.g., TensorFlow Developer, Hugging Face Course, or Generative AI specialization) are a plus.

  • Compliance or governance-related certifications (e.g., SOC 2, ISO 27001) are a differentiator.

KNOWLEDGE AND SKILLS

  • Deep understanding of generative AI architectures, including transformers, diffusion models, and retrieval-augmented generation.

  • Proficiency in Python and ML frameworks such as PyTorch, TensorFlow, and Hugging Face Transformers.

  • Strong knowledge of MLOps practices, including CI/CD pipelines, model versioning, testing, and monitoring.

  • Experience with containerization (Docker, Kubernetes) and orchestration in cloud environments.

  • Solid understanding of data pipelines, Snowflake architecture, and ETL/ELT best practices.

  • Strong knowledge of security, governance, and compliance considerations in AI/ML systems.

  • Experience optimizing AI workloads for cost efficiency without sacrificing performance.

  • Excellent problem-solving skills, ability to make design tradeoffs in complex systems, and mentor junior team members.

  • Strong communication skills to translate technical concepts for business stakeholders and cross-functional teams.

SUPERVISORY RESPONSIBILITIES

This position has no supervisory responsibility but may mentor and train junior engineers.

PHYSICAL REQUIREMENTS

Office environment - no specific or unusual physical or environmental demands and employees are regularly required to sit, walk, stand, talk, and hear. Employees are required to reach with hands and arms; stoop, kneel, crouch, or crawl. Employees must occasionally lift and/or move up to 30 pounds. Employees are required to have visual acuity and be capable of operating and viewing computers and other electronic devices for extended periods of time.

WORK ENVIRONMENT

This position operates in an office environment and requires the frequent use of a computer, telephone, copier, and other standard office equipment.

This description is a general statement of essential job functions and responsibilities.
The position may include other duties as assigned.

We are currently not offering employment sponsorship for this opportunity.

#LI-TP1

#LI-HYBRID

The current range for this position is

$139,874.00 - $250,377.40

This range is exclusive of fringe benefits and potential bonuses. If hired at ICW Group, your final base salary compensation will be determined by factors unique to each candidate, including experience, education and the location of the role and considers employees performing substantially similar work.

WHY JOIN ICW GROUP?

  • Challenging work and the ability to make a difference

  • You will have a voice and feel a sense of belonging

  • We offer a competitive benefits package, with generous medical, dental, and vision plans as well as 401K retirement plans and company match

  • Bonus potential for all positions

  • Paid Time Off

  • Paid holidays throughout the calendar year

  • Want to continue learning? We'll support you 100%


ICW Group is committed to creating a diverse environment and is proud to be an Equal Opportunity Employer. ICW Group will not discriminate against an applicant or employee on the basis of race, color, religion, national origin, ancestry, sex/gender, age, physical or mental disability, military or veteran status, genetic information, sexual orientation, gender identity, gender expression, marital status, or any other characteristic protected by applicable federal, state or local law.

___________________

Job Category

IT