ZipRecruiter

Log In

1

Generative Ai Testing Jobs in Ottawa, ON (NOW HIRING)

Accenture

AI Full Stack Developer

Ottawa, ON

CA$114K - CA$171K/yr

Design, develop, and implement Generative AI agents to enhance user experience and automate ... Establish and uphold coding standards, testing protocols, and quality assurance processes across ...

Ciena

Hardware Characterization Engineer

Ottawa, ON · On-site

CA$118/hr

Experience leveraging AI and Generative AI tools (e.g., coding assistants, automation agents, and LLM-based solutions) to accelerate software development, automate testing, analyze results, and ...

All JobsGenerative Ai Testing JobsGenerative Ai Testing Jobs in Ottawa, ON

Generative Ai Testing information

What is generative AI testing?

Generative AI Testing refers to the process of evaluating and validating AI systems, particularly those that generate content such as text, images, or code. This type of testing focuses on assessing the accuracy, reliability, fairness, and safety of generative models to ensure they function as intended and avoid producing harmful or biased outputs. Testers use various methods, including automated and manual techniques, to check for issues like hallucinations, inappropriate content, or security vulnerabilities. The goal is to build trust in generative AI systems and ensure they meet quality and ethical standards before deployment.

What are some common challenges faced when testing generative AI models, and how can I prepare to address them in this role?

Testing generative AI models often involves unique challenges such as evaluating the quality and relevance of generated content, detecting bias or inappropriate outputs, and ensuring model consistency across various prompts. You may work closely with data scientists and engineers to create robust evaluation frameworks and develop automated as well as manual testing strategies. Familiarity with prompt engineering, statistical evaluation techniques, and domain-specific knowledge will help you address these challenges effectively. Proactively staying updated on industry best practices and collaborating with cross-functional teams are key to success in this dynamic field.

What are the key skills and qualifications needed to thrive as a generative AI testing specialist, and why are they important?

To thrive as a Generative AI Testing Specialist, you need a robust understanding of machine learning principles, model evaluation techniques, and a background in computer science or a related field. Familiarity with tools such as Python, TensorFlow, PyTorch, and model evaluation frameworks, as well as experience with automated testing platforms, is typically required. Analytical thinking, attention to detail, and strong communication skills help you identify model weaknesses and collaborate effectively with development teams. These skills are crucial to ensure the reliability, safety, and ethical deployment of generative AI solutions.

What is the difference between Generative Ai Testing vs Data Scientist?

AspectGenerative Ai TestingData Scientist
Required CredentialsKnowledge of AI models, testing tools, programming skillsStatistics, programming, data analysis certifications
Work EnvironmentAI development teams, testing labs, tech companiesResearch labs, tech firms, finance, healthcare
Employer & Industry UsageAI product testing, quality assurance in techData analysis, predictive modeling across industries

Generative Ai Testing focuses on evaluating and validating AI-generated content and models, ensuring quality and accuracy. Data Scientists analyze data, build models, and derive insights. While both roles require programming and AI knowledge, Generative Ai Testing emphasizes testing processes, whereas Data Scientists focus on data analysis and model development.

How do I become a Generative AI Testing?

To become a Generative AI Tester, develop skills in machine learning, natural language processing, and programming languages like Python. Gain experience with AI frameworks such as TensorFlow or PyTorch and understand data quality and model evaluation techniques. Relevant certifications and hands-on projects can enhance your qualifications for roles in AI testing environments.

Is Generative AI Testing a good career?

Generative AI Testing is a growing field within AI development, focusing on evaluating the quality and safety of AI-generated content. It requires skills in machine learning, programming, and understanding AI models, often involving tools like Python and TensorFlow. The role offers opportunities in tech companies and research labs, with demand expected to increase as AI applications expand.

What are popular job titles related to Generative Ai Testing jobs in Ottawa, ON?

For Generative Ai Testing jobs in Ottawa, ON, the most frequently searched job titles are:

What job categories do people searching Generative Ai Testing jobs in Ottawa, ON look for?

The top searched job categories for Generative Ai Testing jobs in Ottawa, ON are:

Generative Ai Testing jobs near you
Infographic showing various Generative Ai Testing job openings in Ottawa, ON as of August 2026, with employment types broken down into 1% As Needed, 78% Full Time, 16% Part Time, 4% Contract, and 1% Nights. Highlights an 87% Physical, 2% Hybrid, and 11% Remote job distribution.

AI Engineer - Generative & Agentic AI

Ottawa, ON • On-site

BDO Canada
See more jobs at BDO Canadaexternal

CA$84K - CA$128K/yr

Full-time

PTO

Re-posted 15 days ago

Job description

Putting people first, every day

BDO is a firm built on a foundation of positive relationships with our people and our clients. Each day, our professionals provide exceptional service, helping clients with advice and insight they can trust. In turn, we offer an award-winning environment that fosters apeople-first culturewith a high priority on your personal and professional growth.

Your Opportunity

We are seeking an experienced AI Engineer with a strong foundation in software engineering and a proven track record of building Generative and Agentic AI solutions. This new role is ideal for someone who thrives at the intersection of software development, applied AI research, and practical solution design. You will be responsible for designing, implementing, and deploying production-grade AI systems that leverage the latest in large language models (LLMs), orchestration frameworks, and agentic architectures.

Key Responsibilities

  • Design & Development
    • Build scalable and robust AI-driven applications using LLMs and agentic frameworks.
    • Implement vector database solutions to enable semantic search, RAG (retrieval augmented generation), and knowledge management.
    • Develop orchestration pipelines with frameworks like LangChain, Semantic Kernel, AutoGen, or emerging agentic frameworks such as Agno and Parlant.
  • Automation & Integration
    • Integrate workflow automation platforms (n8n, dify.ai Power Automate, etc.) with AI systems to deliver end-to-end business solutions.
    • Ensure seamless integration with existing enterprise software ecosystems and APIs.
  • Quality & Evaluation
    • Apply or contribute to LLM evaluation methodologies, leveraging platforms like Arize, DeepEval, or IBM WatsonX for model benchmarking, monitoring, and continuous improvement.
    • Establish best practices for testing, performance tuning, and evaluation of agentic/LLM-based workflows.
  • Collaboration & Delivery
    • Work closely with cross-functional teams including data scientists, software engineers, and product owners.
    • Translate business requirements into technical designs and deliver working AI-powered applications in production environments.

Required Skills & Qualifications

  • Strong background in software engineering (e.g., system design, distributed systems, APIs, CI/CD).
  • Proven experience in building Generative AI and/or agentic systems.
  • Hands-on expertise with:
    • LLMs and model fine-tuning or integration.
    • Vector databases (e.g., Pinecone, Weaviate, Milvus, FAISS, etc.).
    • LLM orchestration frameworks (LangChain, Semantic Kernel, AutoGen, etc.).
    • Agentic frameworks (Pydantic AI, Agno, Parlant, or similar).
  • Proficiency in at least one major programming language such as Python or .NET (strong engineering fundamentals matter more than specific syntax).
  • Hands-on experience architecting, deploying, and operating cloud-native solutions on one or more major hyperscale platforms (Azure, AWS, or GCP)
  • Experience with workflow automation tools like n8n, dify.ai, Power Automate, or similar platforms.

Nice-to-Have Skills

  • Familiarity with LLM evaluation frameworks (Arize, DeepEval, IBM WatsonX).
  • Contributions to open-source AI frameworks or communities.
  • Experience in MLOps, monitoring, and model deployment pipelines.

Who You Are

  • A problem-solver with strong engineering rigor and curiosity for cutting-edge AI.
  • Comfortable moving between architecture, coding, and deployment.
  • Passionate about transforming abstract business needs into intelligent, production-ready systems.

Salary range for this role is between $84,000 and $128,000.


Why BDO?
Our people-first approach to talent has earned us a spot among Canada's Top 100 Employers for 2026. This recognition is a milestone we're thrilled to add to our collection of awards for both experienced and student talent experiences.


At BDO, our people experience is guided by three core pillars-Do work with genuine care, Do what matters with purpose, and Do what's next - shaping how we support our people, serve our clients, and grow together.

Our firm is committed to providing an environment where you can be successful in the following ways:

  • We enable you to engage with how we change and evolve, being a key contributor to the success and growth of BDO in Canada.
  • We help you become a better professional within our services, industries, and markets with extensive opportunities for learning and development.
  • We support your achievement of personal goals outside of the office and making an impact on your community.
  • We foster a collaborative, inclusive environment where your ideas are valued, and you can do your best work with genuine care and purpose
  • We encourage innovation and forward thinking, empowering you to embrace what's next and help shape the future of our firm

Giving back adds up:Where company meets community.BDO is actively involved in our communities by supporting local charity initiatives. We support staff with local and national events where you will be given the opportunity to contribute to your community.

Total rewards that matter: We pay for performance with competitive total cash compensation that recognizes and rewards your contribution. We provide comprehensive benefits from day one, and a flexible personal time off policy. We're committed to supporting your overall wellbeing and provide reimbursement for wellness initiatives that fit your lifestyle.

Everyone counts: We are committed to creating a workplace where employees can participate fully, contribute meaningfully and succeed without barriers. We are dedicated to fostering a workplace defined by respect, fairness, and a true sense of belonging for everyone. We recognize and celebrate the unique experiences, identities, and perspectives that each of us bring - and that these experiences strengthen how we work together. Our commitment extends to ensuring that our application process is both inclusive and accessible. If you require accommodation to complete the application process, please contact us.


Flexibility: All BDO personnel are expected to spend some of their time working in the office, at the client site, and virtually unless accommodations or alternative work arrangements are in place.

Our model is a blended approach designed to support the flexible needs of our people, the firm and our clients. It's about creating work experiences that meet everyone's needs and providing flexibility to adjust when, where and how we work to meet the expectations of our role.

Code of Conduct: Our Code of Conduct sets clear standards for how we conduct business. It reflects our shared values and commitments and includes guiding principles to help us make ethical decisions and maintain trust with each other, our clients, and the public.

BDO may use artificial intelligence enabled tools to support certain aspects of the recruitment process. While these tools assist our teams, our use of AI does not replace human decision making, and all employment-related outcomes are made by BDO personnel.

More information on BDO Canada's Privacy Policy can be found here:

Privacy Policy | BDO Canada


Ready to make your mark at BDO? Click "Apply now" to send your up-to-date resume to one of our Talent Acquisition Specialists.


To explore other opportunities at BDO, check out ourcareers page.


#LI-MM1

Apply