1

Generative Ai Testing Jobs in San Rafael, CA (NOW HIRING)

... testing, and CI/CD for ML services. * Collaborate closely with Forward Deployed Engineers, ML ... Experience integrating ML or Generative AI models (LLMs, multimodal models) into backend services ...

Lead Data Scientist

San Francisco, CA · On-site +1

$130K - $200K/yr

Own Generative AI Analytics: Build and maintain end-to-end infra (semantic layer, LLM tooling ... Experience with Streamlit, Snowflake Cortex, AI/data labeling, A/B testing, and two-sided ...

Your Role: * Develop and test AI/ML and Generative AI solutions under guidance from senior ... testing. * Integrate AI models/APIs into enterprise applications and workflows. * Participate in ...

Utilize rigorous testing and iterative improvement processes to optimize the performance and ... Experience with LLMs and imaginative generative AI. * Participation in research projects/papers ...

Research Engineer

San Francisco, CA · On-site

$120K - $200K/yr

Utilize rigorous testing and iterative improvement processes to optimize the performance and ... Experience with LLMs and imaginative generative AI. * Participation in research projects/papers ...

Senior Quality Engineer

San Francisco, CA

$104K - $141K/yr

You will document AI-testing standards and formally mentor the team on integrating Generative AI into their daily engineering workflows. Scope, Complexity, and Impact * Scope: You will lead quality ...

Senior Quality Engineer

San Francisco, CA · On-site

$104K - $141K/yr

You will document AI-testing standards and formally mentor the team on integrating Generative AI into their daily engineering workflows. Scope, Complexity, and Impact * Scope: You will lead quality ...

Senior Quality Engineer

San Francisco, CA · On-site

$104K - $141K/yr

You will document AI-testing standards and formally mentor the team on integrating Generative AI into their daily engineering workflows. Scope, Complexity, and Impact * Scope: You will lead quality ...

Showing results 21-40

Generative Ai Testing information

See San Rafael, CA salary details

$35

$59

$85

How much do generative ai testing jobs pay per hour?

As of Sep 14, 2026, the average hourly pay for generative ai testing in San Rafael, CA is $59.89, according to ZipRecruiter salary data. Most workers in this role earn between $49.33 and $68.61 per hour, depending on experience, location, and employer.

What is generative AI testing?

Generative AI Testing refers to the process of evaluating and validating AI systems, particularly those that generate content such as text, images, or code. This type of testing focuses on assessing the accuracy, reliability, fairness, and safety of generative models to ensure they function as intended and avoid producing harmful or biased outputs. Testers use various methods, including automated and manual techniques, to check for issues like hallucinations, inappropriate content, or security vulnerabilities. The goal is to build trust in generative AI systems and ensure they meet quality and ethical standards before deployment.

What are some common challenges faced when testing generative AI models, and how can I prepare to address them in this role?

Testing generative AI models often involves unique challenges such as evaluating the quality and relevance of generated content, detecting bias or inappropriate outputs, and ensuring model consistency across various prompts. You may work closely with data scientists and engineers to create robust evaluation frameworks and develop automated as well as manual testing strategies. Familiarity with prompt engineering, statistical evaluation techniques, and domain-specific knowledge will help you address these challenges effectively. Proactively staying updated on industry best practices and collaborating with cross-functional teams are key to success in this dynamic field.

What are the key skills and qualifications needed to thrive as a generative AI testing specialist, and why are they important?

To thrive as a Generative AI Testing Specialist, you need a robust understanding of machine learning principles, model evaluation techniques, and a background in computer science or a related field. Familiarity with tools such as Python, TensorFlow, PyTorch, and model evaluation frameworks, as well as experience with automated testing platforms, is typically required. Analytical thinking, attention to detail, and strong communication skills help you identify model weaknesses and collaborate effectively with development teams. These skills are crucial to ensure the reliability, safety, and ethical deployment of generative AI solutions.

What is the difference between Generative Ai Testing vs Data Scientist?

AspectGenerative Ai TestingData Scientist
Required CredentialsKnowledge of AI models, testing tools, programming skillsStatistics, programming, data analysis certifications
Work EnvironmentAI development teams, testing labs, tech companiesResearch labs, tech firms, finance, healthcare
Employer & Industry UsageAI product testing, quality assurance in techData analysis, predictive modeling across industries

Generative Ai Testing focuses on evaluating and validating AI-generated content and models, ensuring quality and accuracy. Data Scientists analyze data, build models, and derive insights. While both roles require programming and AI knowledge, Generative Ai Testing emphasizes testing processes, whereas Data Scientists focus on data analysis and model development.

How do I become a Generative AI Testing?

To become a Generative AI Tester, develop skills in machine learning, natural language processing, and programming languages like Python. Gain experience with AI frameworks such as TensorFlow or PyTorch and understand data quality and model evaluation techniques. Relevant certifications and hands-on projects can enhance your qualifications for roles in AI testing environments.

Is Generative AI Testing a good career?

Generative AI Testing is a growing field within AI development, focusing on evaluating the quality and safety of AI-generated content. It requires skills in machine learning, programming, and understanding AI models, often involving tools like Python and TensorFlow. The role offers opportunities in tech companies and research labs, with demand expected to increase as AI applications expand.

What cities near San Rafael, CA are hiring for Generative Ai Testing jobs?

Cities near San Rafael, CA with the most Generative Ai Testing job openings:

Infographic showing various Generative Ai Testing job openings in San Rafael, CA as of June 2026, with employment types broken down into 91% Full Time, 5% Part Time, and 4% Contract. Highlights an 66% Physical, 3% Hybrid, and 31% Remote job distribution, with an average salary of $124,569 per year, or $59.9 per hour.

Agentic AI Engineer, Senior - Anthropic/Claude

San Francisco, CA • On-site

Deloitte
Finance and Insurance • 10K+ employees

$123K - $169K/yr

Full-time

Posted 24 days ago


Key responsibilities

  • Design and build Generative AI and agentic solutions using Claude, including agent orchestration, tool integration, and Model Context Protocol (MCP) based integrations.

  • Manage and deliver components of client engagements spanning solution design, prototyping, testing, and production rollout.

  • Lead small teams through business requirements gathering, functional design, process mapping, and support-model definition, providing technical direction throughout.


Deloitte rating

8.2

Company rating: 8.2 out of 10

Based on 93 frontline employees who took The Breakroom Quiz


Job description

Agentic AI is moving from experimentation to production, and organizations everywhere are racing to figure out how to build it responsibly and at scale. We're growing a team of engineers who want to work at the center of that shift: designing and building autonomous, multi-agent systems powered by Anthropic's Claude models for some of the world's largest organizations.

Join our AI & Engineering team in transforming technology platforms, driving innovation, and helping make a significant impact on our clients' success. You'll work alongside talented professionals reimagining and re-engineering operations and processes that are critical to businesses. Your contributions can help clients improve financial performance, accelerate new digital ventures, and fuel growth through innovation.

AI & Engineering leverages cutting-edge engineering capabilities to build, deploy, and operate integrated/verticalized sector solutions in software, data, AI, network, and hybrid cloud infrastructure. These solutions are powered by engineering for business advantage, transforming mission-critical operations. We enable clients to stay ahead with the latest advancements by transforming engineering teams and modernizing technology & data platforms. Our delivery models are tailored to meet each client's unique requirements.

Recruiting for this role ends on 09/30/2026.

Work you'll do

As an AI and Data Science Engineer III on the AI & Data team, you will be responsible for driving technology-focused client delivery across complex engagements.

  • Manage day-to-day interactions with client stakeholders and sponsors, acting as the primary technical point of contact for your workstream
  • Design and build Generative AI and agentic solutions using Claude, including agent orchestration, tool integration, and Model Context Protocol (MCP) based integrations, using the Claude Agent SDK and Claude Cowork
  • Manage and deliver components of client engagements spanning solution design, prototyping, testing, and production rollout
  • Lead small teams through business requirements gathering, functional design, process mapping, and support-model definition, providing technical direction throughout
  • Develop and maintain project scope, schedule, and resource plans for your workstream, and adjust them as engagement needs evolve
  • Monitor progress against plan, identify variances, and drive corrective action before issues affect delivery
  • Manage changes to scope, schedule, and cost in line with the engagement's change management process
  • Identify risks, assumptions, and constraints early, and implement approved mitigations to protect delivery
  • Maintain clear, consistent communication with client stakeholders and internal engagement leadership

You'll be deployed onto client engagements that typically involve one or more of the following:

  • Designing and building agentic AI solutions powered by Claude, from proof-of-concept through production deployment
  • Implementing multi-agent systems that automate business processes, integrate with enterprise tools, and operate within defined autonomy and guardrails
  • Modernizing existing AI/ML or data platforms to incorporate Generative AI capabilities, including retrieval-augmented generation and agent-based orchestration
  • Advising clients on Generative AI adoption strategy, responsible AI practices, and model/platform selection
  • Partnering directly with client technology, data, and business stakeholders to scope and deliver AI solutions

A successful candidate would possess these skills:

  • Ability to work independently and collaborate as part of a team
  • Effective written and verbal communication skills
  • Meticulous attention to detail and quality of work product
  • Ability to build and sustain professional relationships 
  • Ability to lead projects or workstreams
  • Ability to manage and prioritize multiple tasks in a fast-paced and dynamic environment
  • Strong interpersonal skills and professional demeanor 
  • Ability to meet deadlines
  • Ability to provide clear guidance to others

The team

Our AI & Data practice offers comprehensive solutions for designing, developing, and operating advanced Data and AI platforms, products, insights, and services. We help clients innovate, enhance, and manage their data, AI, and analytics capabilities, ensuring they can grow and scale effectively.

Qualifications

Required:

  • 4+ years of experience designing, developing, and delivering production AI/ML solutions, including at least 1 year delivering production Generative AI or Agentic AI systems
  • 2+ years of hands-on experience developing AI/ML applications in Python
  • At least 1 year of direct experience designing or implementing production agentic workflows involving capabilities such as orchestration, tool integration, state management, evaluation, guardrails, and human-in-the-loop controls
  • At least 1 year of hands-on experience developing agentic AI solutions using the Claude Agent SDK, including agent orchestration, tool integration, permissions, or MCP integrations
  • At least 1 year of hands-on experience developing and deploying production solutions using Anthropic Claude models through the native Anthropic API or a supported cloud platform
  • At least 1 year of experience applying prompt engineering, context engineering, structured outputs, and related techniques to improve the performance, reliability, and usability of AI solutions
  • 1+ years of experience leading project workstreams or engagements, translating business problems into AI solutions, and delivering measurable outcomes
  • Bachelor's or Master's degree in Computer Science, Engineering, Data Science, AI, or a related field
  • Ability to travel up to 50% on average, based on the work you do and the clients and industries/sectors you serve
  • Limited immigration sponsorship may be available

Preferred:

  • Experience in consulting or other client-facing delivery roles
  • Experience using Claude Cowork to support enterprise workflows, automation, research, or other knowledge-work use cases
  • Hands-on experience using Claude through one or more of the following: Azure AI Foundry, Amazon Bedrock, Google Vertex AI, or the native Anthropic API
  • Experience with multi-agent systems, MCP integrations, agent evaluation, observability, safety controls, or production governance
  • Certifications such as Claude Certified Architect - Professional (CCAR-P), Claude Certified Architect - Foundations (CCAR-F), or Claude Certified Developer - Foundations (CCDV-F)
  • Cloud certification(s) - AWS, Azure, or Google Cloud
  • Experience creating client workshop collateral and leading interactive working sessions
  • Experience presenting technical content to both large and small audiences

The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs. The disclosed range estimate has not been adjusted for the applicable geographic differential associated with the location at which the position may be filled. At Deloitte, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range is $122,000 to $240,500.

You may also be eligible to participate in a discretionary annual incentive program, subject to the rules governing the program, whereby an award, if any, depends on various factors, including, without limitation, individual and organizational performance.

Qualifications:

Agentic AI is moving from experimentation to production, and organizations everywhere are racing to figure out how to build it responsibly and at scale. We're growing a team of engineers who want to work at the center of that shift: designing and building autonomous, multi-agent systems powered by Anthropic's Claude models for some of the world's largest organizations.

Join our AI & Engineering team in transforming technology platforms, driving innovation, and helping make a significant impact on our clients' success. You'll work alongside talented professionals reimagining and re-engineering operations and processes that are critical to businesses. Your contributions can help clients improve financial performance, accelerate new digital ventures, and fuel growth through innovation.

AI & Engineering leverages cutting-edge engineering capabilities to build, deploy, and operate integrated/verticalized sector solutions in software, data, AI, network, and hybrid cloud infrastructure. These solutions are powered by engineering for business advantage, transforming mission-critical operations. We enable clients to stay ahead with the latest advancements by transforming engineering teams and modernizing technology & data platforms. Our delivery models are tailored to meet each client's unique requirements.

Recruiting for this role ends on 09/30/2026.

Work you'll do

As an AI and Data Science Engineer III on the AI & Data team, you will be responsible for driving technology-focused client delivery across complex engagements.

  • Manage day-to-day interactions with client stakeholders and sponsors, acting as the primary technical point of contact for your workstream
  • Design and build Generative AI and agentic solutions using Claude, including agent orchestration, tool integration, and Model Context Protocol (MCP) based integrations, using the Claude Agent SDK and Claude Cowork
  • Manage and deliver components of client engagements spanning solution design, prototyping, testing, and production rollout
  • Lead small teams through business requirements gathering, functional design, process mapping, and support-model definition, providing technical direction throughout
  • Develop and maintain project scope, schedule, and resource plans for your workstream, and adjust them as engagement needs evolve
  • Monitor progress against plan, identify variances, and drive corrective action before issues affect delivery
  • Manage changes to scope, schedule, and cost in line with the engagement's change management process
  • Identify risks, assumptions, and constraints early, and implement approved mitigations to protect delivery
  • Maintain clear, consistent communication with client stakeholders and internal engagement leadership

You'll be deployed onto client engagements that typically involve one or more of the following:

  • Designing and building agentic AI solutions powered by Claude, from proof-of-concept through production deployment
  • Implementing multi-agent systems that automate business processes, integrate with enterprise tools, and operate within defined autonomy and guardrails
  • Modernizing existing AI/ML or data platforms to incorporate Generative AI capabilities, including retrieval-augmented generation and agent-based orchestration
  • Advising clients on Generative AI adoption strategy, responsible AI practices, and model/platform selection
  • Partnering directly with client technology, data, and business stakeholders to scope and deliver AI solutions

A successful candidate would possess these skills:

  • Ability to work independently and collaborate as part of a team
  • Effective written and verbal communication skills
  • Meticulous attention to detail and quality of work product
  • Ability to build and sustain professional relationships 
  • Ability to lead projects or workstreams
  • Ability to manage and prioritize multiple tasks in a fast-paced and dynamic environment
  • Strong interpersonal skills and professional demeanor 
  • Ability to meet deadlines
  • Ability to provide clear guidance to others

The team

Our AI & Data practice offers comprehensive solutions for designing, developing, and operating advanced Data and AI platforms, products, insights, and services. We help clients innovate, enhance, and manage their data, AI, and analytics capabilities, ensuring they can grow and scale effectively.

Qualifications

Required:

  • 4+ years of experience designing, developing, and delivering production AI/ML solutions, including at least 1 year delivering production Generative AI or Agentic AI systems
  • 2+ years of hands-on experience developing AI/ML applications in Python
  • At least 1 year of direct experience designing or implementing production agentic workflows involving capabilities such as orchestration, tool integration, state management, evaluation, guardrails, and human-in-the-loop controls
  • At least 1 year of hands-on experience developing agentic AI solutions using the Claude Agent SDK, including agent orchestration, tool integration, permissions, or MCP integrations
  • At least 1 year of hands-on experience developing and deploying production solutions using Anthropic Claude models through the native Anthropic API or a supported cloud platform
  • At least 1 year of experience applying prompt engineering, context engineering, structured outputs, and related techniques to improve the performance, reliability, and usability of AI solutions
  • 1+ years of experience leading project workstreams or engagements, translating business problems into AI solutions, and delivering measurable outcomes
  • Bachelor's or Master's degree in Computer Science, Engineering, Data Science, AI, or a related field
  • Ability to travel up to 50% on average, based on the work you do and the clients and industries/sectors you serve
  • Limited immigration sponsorship may be available

Preferred:

  • Experience in consulting or other client-facing delivery roles
  • Experience using Claude Cowork to support enterprise workflows, automation, research, or other knowledge-work use cases
  • Hands-on experience using Claude through one or more of the following: Azure AI Foundry, Amazon Bedrock, Google Vertex AI, or the n...

What Deloitte employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom