ZipRecruiter

Log In

1

Ai Qa Jobs in Quebec (NOW HIRING)

Jestais

Training or exposure to AI-assisted testing tools or frameworks (preferred) * Agile or Scrum certification (preferred) Knowledge & Experience * 3-5 years of experience in Quality Assurance, with a ...

Jestais

Stay current with emerging testing methodologies, automation tools, AI-assisted testing practices, and enterprise QA trends Qualifications Education & certification * Bachelor's degree in Computer ...

Unitytech

Senior Software Engineer, QA

Montreal, QC · On-site

Senior Software Engineer, Quality The Opportunity We are seeking a Senior Quality Engineer to join ... Fluent with AI tools, understand their limitations, and know when a human needs to be involved You ...

Keywords Studios

Polish QA Proofreader

Montreal, QC · On-site

Candidates should refrain from using AI assistance or third-party tools or services that could ... Americas, Canada, Montreal Area of Work: QA Testing Services Service: Globalize Employment Type ...

Side

Microservices AI Developer

Montreal, QC

... quality assurance, localization QA, player support, community management, and datasets. Help us ... For more information, visit www.side.inc About This Opportunity As an AI Microservices Developer ...

YO AI Labs

Lawyer - Remote

Montreal, QC · Remote

$140 - $350/hr

Experience with legal content review or quality assurance. * Familiarity with rubric-based evaluation of legal documents is a plus. * Interest in legal technology, AI, or emerging technologies is a ...

next page

Showing results 1-20

All JobsAi Qa JobsAi Qa Jobs in Quebec

Ai Qa information

What is an AI QA?

AI QA (Artificial Intelligence Quality Assurance) professionals are specialists who test and validate machine learning models, AI systems, and related software to ensure they function correctly, reliably, and ethically. They design test cases, evaluate model accuracy, check for biases, and verify compliance with regulatory standards. AI QA experts work closely with data scientists, developers, and product managers to identify issues and improve system performance throughout the AI development lifecycle.

What are the key skills and qualifications needed to thrive as an AI QA?

To thrive as an AI QA specialist, you need strong analytical skills, knowledge of software testing methodologies, and a solid understanding of AI/ML concepts, typically supported by a degree in computer science or a related field. Familiarity with testing frameworks like Selenium or PyTest, version control systems like Git, and tools such as TensorFlow or PyTorch for model validation is often required. Attention to detail, problem-solving abilities, and effective communication set top performers apart in this role. These skills ensure the reliability, fairness, and safety of AI systems, which is critical for delivering trustworthy and high-quality AI solutions.

How does an AI QA engineer typically collaborate with data scientists and developers during the model development lifecycle?

AI QA engineers work closely with data scientists and developers throughout the AI model development process. They often participate in sprint planning and daily stand-up meetings to align on testing strategies, clarify requirements, and address potential model issues early. Their responsibilities include designing test cases for model accuracy, fairness, and robustness, as well as validating data pipelines and deployment workflows. Effective communication and a proactive approach are essential, as AI QA engineers help ensure the final model meets quality standards and integrates smoothly into production environments.

What is the difference between Ai Qa vs Data Analyst?

AspectAi QaData Analyst
Required CredentialsTypically certifications in AI, QA, or software testingBachelor's degree in data science, statistics, or related fields
Work EnvironmentTech companies, software development teams, AI-focused projectsBusiness, finance, healthcare, and various industries analyzing data
Employer & Industry UsageTech firms, AI startups, software companiesCorporations across multiple sectors, consulting firms
Common Search & Comparison IntentUnderstanding roles in AI quality assuranceAnalyzing data to inform business decisions

Ai Qa specialists focus on testing and ensuring the quality of AI systems, often requiring knowledge of AI tools and testing protocols. Data Analysts interpret data to generate insights, requiring skills in data visualization and statistical analysis. While both roles work with data and technology, Ai Qa emphasizes quality assurance in AI development, whereas Data Analysts focus on data interpretation for decision-making.

How to become an AI QA tester?

To become an AI QA tester, you should have a strong understanding of software testing principles, familiarity with AI and machine learning concepts, and experience with testing tools and scripting languages. Gaining knowledge of AI models, data validation, and quality assurance processes is essential, and certifications in software testing or AI can enhance your qualifications.

What are popular job titles related to Ai Qa jobs in Quebec?

For Ai Qa jobs in Quebec, the most frequently searched job titles are:

What job categories do people searching Ai Qa jobs in Quebec look for?

The top searched job categories for Ai Qa jobs in Quebec are:

What cities in Quebec are hiring for Ai Qa jobs?

Cities in Quebec with the most Ai Qa job openings:

Ai Qa jobs near you
Infographic showing various Ai Qa job openings in Quebec as of August 2026, with employment types broken down into 50% Full Time, 18% Part Time, and 32% Contract. Highlights an 77% In-person, and 23% Remote job distribution.

QA Lead - AI Systems & Models Testing

Jay Analytix

Montreal, QC

Contractor

Re-posted 26 days ago

Job description

QA Lead - AI Systems & Models Testing

Quality Assurance Artificial Intelligence Contract Position

Contract

Montreal, QC

AI / ML Testing

LLM / RAG / LangChain

ABOUT THE ROLE

We are seeking an experienced QA Lead with deep expertise in AI systems testing to join our team on a contract basis in Montreal, Quebec. This role sits at the intersection of quality engineering and artificial intelligence, requiring hands-on proficiency in LLM behavior analysis, RAG pipeline validation, and modern AI orchestration frameworks. You will own the end-to-end test strategy for complex AI products and help define quality standards in a rapidly evolving space.

MUST-HAVE SKILLS

  • Proven QA leadership experience designing and executing test strategies for AI/ML systems or LLM-powered applications.
  • Strong understanding of LLM internals: tokenization, embeddings, attention mechanisms, and inference behavior to anticipate and diagnose failure modes.
  • Hands-on experience with prompt engineering - constructing effective prompts, detecting hallucinations, and evaluating outputs across accuracy, tone, coherence, and bias dimensions.
  • Experience testing RAG pipelines and knowledge base integrations, including validation of data quality and retrieval accuracy as they impact model outputs.
  • Familiarity with vector database mechanics: similarity search thresholds, embedding drift, near-duplicate documents, and sparse vs. dense embeddings.
  • Practical experience with LangChain and/or LangGraph - able to read chain/graph construction code, identify failure points, and write test harnesses.
  • Ability to validate MCP (Model Context Protocol) integration points, including tool availability and error-handling scenarios.
  • Proficiency applying generative AI evaluation metrics and establishing quality thresholds appropriate for production AI systems.
  • Excellent written and verbal communication in English; bilingualism (English/French) is a plus for the Montreal market.

NICE-TO-HAVE SKILLS

  • Experience with bias detection and safety testing frameworks for AI systems.
  • Exposure to performance and scalability testing of vector databases under high load.
  • Familiarity with CI/CD pipelines for ML model deployment and automated regression testing.
  • Knowledge of responsible AI principles and AI governance frameworks.
  • Contributions to or experience with open-source AI testing or evaluation tooling (e.g., DeepEval, Ragas, PromptFlow).
  • Background in data engineering or data quality practices relevant to AI pipeline inputs.
  • Cloud platform experience (AWS, Azure, or GCP) in the context of deploying or testing AI workloads.

KEY RESPONSIBILITIES

  • Lead design and execution of comprehensive test strategies across AI systems, including prompt evaluation, output quality assessment, and bias/safety analysis.
  • Develop and maintain test harnesses for LangChain and LangGraph-based applications; review chain and graph construction code to proactively surface integration risks.
  • Validate RAG pipeline integrity - data ingestion, chunking, retrieval accuracy, and embedding consistency - and define edge-case coverage for vector database interactions.
  • Establish and track generative AI quality metrics and thresholds; report on model output quality across multiple evaluation dimensions.
  • Collaborate with ML engineers, data scientists, and product teams to embed quality practices throughout the AI development lifecycle.
  • Document test findings clearly for both technical and non-technical stakeholders.

Contract position based in Montreal, Quebec, Canada On-site / Hybrid

Employment Type: CONTRACTOR

Apply