ZipRecruiter

Log In

1

Generative Ai Testing Jobs in Quebec (NOW HIRING)

Fiera Capital Corp.

AI Developer

Montreal, QC · On-site

Lead the architecture, design, and delivery of Generative AI solutions-including multi-step ... testing, reliability, reusability, and secure-by-design delivery * Define evaluation frameworks ...

Side

Microservices AI Developer

Montreal, QC

Understand and become the company's expert in LLM and Generative AI API methods and Open Weights ... Develop tools and utilities to support AI development and testing. * Proactively identify and ...

Side

Microservices AI Developer

Montreal, QC

Understand and become the company's expert in LLM and Generative AI API methods and Open Weights ... Develop tools and utilities to support AI development and testing. * Proactively identify and ...

Jestais

Large Language Models & Generative AI * Design and implement LLM-powered applications using RAG ... and hypothesis testing. * Advanced SQL for enterprise-scale analytics. Soft Skills * Highly ...

Jesta I.S.

You will work at the intersection of predictive modelling, generative AI, and enterprise software ... Design and execute model experiments, hypothesis testing, oracle testing, and statistical ...

Groupe DCM

Aider à l'intégration de modèles d'IA générative (ChatGPT, Claude, Copilot ou autres ... Collaborer avec les équipes pour tester et améliorer les outils développés. Exigences d'emploi ...

Groupe DCM

Aider à l'intégration de modèles d'IA générative (ChatGPT, Claude, Copilot ou autres ... Collaborer avec les équipes pour tester et améliorer les outils développés. Exigences d'emploi ...

Groupe DCM

Aider à l'intégration de modèles d'IA générative (ChatGPT, Claude, Copilot ou autres ... Collaborer avec les équipes pour tester et améliorer les outils développés. Exigences d'emploi ...

KPMG

... testing, release, and sustainment * Strong communication skills with the ability to engage business ... Exposure to AI agent tools and generative AI capabilities such as Copilot Studio, Azure AI, Azure ...

next page

Showing results 1-20

All JobsGenerative Ai Testing JobsGenerative Ai Testing Jobs in Quebec

Generative Ai Testing information

What is the difference between Generative Ai Testing vs Data Scientist?

AspectGenerative Ai TestingData Scientist
Required CredentialsKnowledge of AI models, testing tools, programming skillsStatistics, programming, data analysis certifications
Work EnvironmentAI development teams, testing labs, tech companiesResearch labs, tech firms, finance, healthcare
Employer & Industry UsageAI product testing, quality assurance in techData analysis, predictive modeling across industries

Generative Ai Testing focuses on evaluating and validating AI-generated content and models, ensuring quality and accuracy. Data Scientists analyze data, build models, and derive insights. While both roles require programming and AI knowledge, Generative Ai Testing emphasizes testing processes, whereas Data Scientists focus on data analysis and model development.

How much do AI testers get paid?

AI testers, involved in evaluating and validating generative AI models, typically earn salaries ranging from $60,000 to $120,000 annually depending on experience, location, and company size. Entry-level positions may start lower, while experienced testers with specialized skills in machine learning and data analysis can earn higher wages.

Is AI testing a good career?

AI testing, including roles like Generative AI Testing, is a growing field with increasing demand for skills in machine learning, data analysis, and programming. It offers opportunities in tech companies, research labs, and startups, often requiring knowledge of AI frameworks and testing tools. The career can be stable and rewarding for those with technical expertise and an interest in AI development and quality assurance.

What are the key skills and qualifications needed to thrive as a Generative AI Testing Specialist, and why are they important?

To thrive as a Generative AI Testing Specialist, you need a robust understanding of machine learning principles, model evaluation techniques, and a background in computer science or a related field. Familiarity with tools such as Python, TensorFlow, PyTorch, and model evaluation frameworks, as well as experience with automated testing platforms, is typically required. Analytical thinking, attention to detail, and strong communication skills help you identify model weaknesses and collaborate effectively with development teams. These skills are crucial to ensure the reliability, safety, and ethical deployment of generative AI solutions.

What are some common challenges faced when testing generative AI models, and how can I prepare to address them in this role?

Testing generative AI models often involves unique challenges such as evaluating the quality and relevance of generated content, detecting bias or inappropriate outputs, and ensuring model consistency across various prompts. You may work closely with data scientists and engineers to create robust evaluation frameworks and develop automated as well as manual testing strategies. Familiarity with prompt engineering, statistical evaluation techniques, and domain-specific knowledge will help you address these challenges effectively. Proactively staying updated on industry best practices and collaborating with cross-functional teams are key to success in this dynamic field.

What is a $900,000 AI job?

A $900,000 AI job typically refers to high-level roles in artificial intelligence, such as senior machine learning engineers, AI research directors, or executive positions, often requiring advanced skills, extensive experience, and sometimes specialized certifications. These roles usually involve leading AI development projects, strategic planning, and overseeing AI teams in large organizations or tech companies.

How do I become an AI tester?

To become an AI tester, you should have a strong understanding of machine learning concepts, programming skills in languages like Python, and experience with data annotation and model evaluation. Familiarity with AI development tools and testing frameworks, along with attention to detail, is essential for identifying issues in AI systems.

What is Generative AI Testing?

Generative AI Testing refers to the process of evaluating and validating AI systems, particularly those that generate content such as text, images, or code. This type of testing focuses on assessing the accuracy, reliability, fairness, and safety of generative models to ensure they function as intended and avoid producing harmful or biased outputs. Testers use various methods, including automated and manual techniques, to check for issues like hallucinations, inappropriate content, or security vulnerabilities. The goal is to build trust in generative AI systems and ensure they meet quality and ethical standards before deployment.
What are popular job titles related to Generative Ai Testing jobs in Quebec? For Generative Ai Testing jobs in Quebec, the most frequently searched job titles are:
What job categories do people searching Generative Ai Testing jobs in Quebec look for? The top searched job categories for Generative Ai Testing jobs in Quebec are:
Generative Ai Testing jobs near you
Infographic showing various Generative Ai Testing job openings in Quebec as of June 2026, with employment types broken down into 100% Full Time. Highlights an 50% In-person, and 50% Remote job distribution.

QA Lead AI Systems & Models Testing

Jay Analytix

Montreal, QC • On-site

Contractor

Posted 17 days ago

Job description

QA Lead AI Systems & Models Testing

Quality Assurance Artificial Intelligence Contract Position

Contract

Montreal, QC

AI / ML Testing

LLM / RAG / LangChain

ABOUT THE ROLE

We are seeking an experienced QA Lead with deep expertise in AI systems testing to join our team on a contract basis in Montreal, Qubec. This role sits at the intersection of quality engineering and artificial intelligence, requiring hands-on proficiency in LLM behavior analysis, RAG pipeline validation, and modern AI orchestration frameworks. You will own the end-to-end test strategy for complex AI products and help define quality standards in a rapidly evolving space.

MUST-HAVE SKILLS

  • Proven QA leadership experience designing and executing test strategies for AI/ML systems or LLM-powered applications.
  • Strong understanding of LLM internals: tokenization, embeddings, attention mechanisms, and inference behavior to anticipate and diagnose failure modes.
  • Hands-on experience with prompt engineering constructing effective prompts, detecting hallucinations, and evaluating outputs across accuracy, tone, coherence, and bias dimensions.
  • Experience testing RAG pipelines and knowledge base integrations, including validation of data quality and retrieval accuracy as they impact model outputs.
  • Familiarity with vector database mechanics: similarity search thresholds, embedding drift, near-duplicate documents, and sparse vs. dense embeddings.
  • Practical experience with LangChain and/or LangGraph able to read chain/graph construction code, identify failure points, and write test harnesses.
  • Ability to validate MCP (Model Context Protocol) integration points, including tool availability and error-handling scenarios.
  • Proficiency applying generative AI evaluation metrics and establishing quality thresholds appropriate for production AI systems.
  • Excellent written and verbal communication in English; bilingualism (English/French) is a plus for the Montreal market.

NICE-TO-HAVE SKILLS

  • Experience with bias detection and safety testing frameworks for AI systems.
  • Exposure to performance and scalability testing of vector databases under high load.
  • Familiarity with CI/CD pipelines for ML model deployment and automated regression testing.
  • Knowledge of responsible AI principles and AI governance frameworks.
  • Contributions to or experience with open-source AI testing or evaluation tooling (e.g., DeepEval, Ragas, PromptFlow).
  • Background in data engineering or data quality practices relevant to AI pipeline inputs.
  • Cloud platform experience (AWS, Azure, or GCP) in the context of deploying or testing AI workloads.

KEY RESPONSIBILITIES

  • Lead design and execution of comprehensive test strategies across AI systems, including prompt evaluation, output quality assessment, and bias/safety analysis.
  • Develop and maintain test harnesses for LangChain and LangGraph-based applications; review chain and graph construction code to proactively surface integration risks.
  • Validate RAG pipeline integrity data ingestion, chunking, retrieval accuracy, and embedding consistency and define edge-case coverage for vector database interactions.
  • Establish and track generative AI quality metrics and thresholds; report on model output quality across multiple evaluation dimensions.
  • Collaborate with ML engineers, data scientists, and product teams to embed quality practices throughout the AI development lifecycle.
  • Document test findings clearly for both technical and non-technical stakeholders.

Contract position based in Montreal, Qubec, Canada On-site / Hybrid