ZipRecruiter

Log In

1

Generative Ai Testing Jobs in New York (NOW HIRING)

Zolon Tech Solutions Inc

Gen AI Analyst

Manhattan, NY · On-site

Working closely with the Generative AI Product Owner/Product Manager, this role will assist in evaluating AI use cases, documenting business requirements, testing AI solutions, and supporting ...

New

Aquent

Gen AI Analyst

Manhattan, NY · On-site

$35 - $48/hr

Working closely with the Generative AI Product Owner/Product Manager, this role will assist in evaluating AI use cases, documenting business requirements, testing AI solutions, and supporting ...

New

Skill

Gen AI Analyst [212408]

Manhattan, NY · On-site

$35 - $48/hr

Working closely with the Generative AI Product Owner/Product Manager, this role will assist in evaluating AI use cases, documenting business requirements, testing AI solutions, and supporting ...

New

previous pagenext page

Showing results 21-40

All JobsGenerative Ai Testing JobsGenerative Ai Testing Jobs in New York

Generative Ai Testing information

What is generative AI testing?

Generative AI Testing refers to the process of evaluating and validating AI systems, particularly those that generate content such as text, images, or code. This type of testing focuses on assessing the accuracy, reliability, fairness, and safety of generative models to ensure they function as intended and avoid producing harmful or biased outputs. Testers use various methods, including automated and manual techniques, to check for issues like hallucinations, inappropriate content, or security vulnerabilities. The goal is to build trust in generative AI systems and ensure they meet quality and ethical standards before deployment.

What are some common challenges faced when testing generative AI models, and how can I prepare to address them in this role?

Testing generative AI models often involves unique challenges such as evaluating the quality and relevance of generated content, detecting bias or inappropriate outputs, and ensuring model consistency across various prompts. You may work closely with data scientists and engineers to create robust evaluation frameworks and develop automated as well as manual testing strategies. Familiarity with prompt engineering, statistical evaluation techniques, and domain-specific knowledge will help you address these challenges effectively. Proactively staying updated on industry best practices and collaborating with cross-functional teams are key to success in this dynamic field.

What are the key skills and qualifications needed to thrive as a generative AI testing specialist, and why are they important?

To thrive as a Generative AI Testing Specialist, you need a robust understanding of machine learning principles, model evaluation techniques, and a background in computer science or a related field. Familiarity with tools such as Python, TensorFlow, PyTorch, and model evaluation frameworks, as well as experience with automated testing platforms, is typically required. Analytical thinking, attention to detail, and strong communication skills help you identify model weaknesses and collaborate effectively with development teams. These skills are crucial to ensure the reliability, safety, and ethical deployment of generative AI solutions.

What is the difference between Generative Ai Testing vs Data Scientist?

AspectGenerative Ai TestingData Scientist
Required CredentialsKnowledge of AI models, testing tools, programming skillsStatistics, programming, data analysis certifications
Work EnvironmentAI development teams, testing labs, tech companiesResearch labs, tech firms, finance, healthcare
Employer & Industry UsageAI product testing, quality assurance in techData analysis, predictive modeling across industries

Generative Ai Testing focuses on evaluating and validating AI-generated content and models, ensuring quality and accuracy. Data Scientists analyze data, build models, and derive insights. While both roles require programming and AI knowledge, Generative Ai Testing emphasizes testing processes, whereas Data Scientists focus on data analysis and model development.

How do I become a Generative AI Testing?

To become a Generative AI Tester, develop skills in machine learning, natural language processing, and programming languages like Python. Gain experience with AI frameworks such as TensorFlow or PyTorch and understand data quality and model evaluation techniques. Relevant certifications and hands-on projects can enhance your qualifications for roles in AI testing environments.

Is Generative AI Testing a good career?

Generative AI Testing is a growing field within AI development, focusing on evaluating the quality and safety of AI-generated content. It requires skills in machine learning, programming, and understanding AI models, often involving tools like Python and TensorFlow. The role offers opportunities in tech companies and research labs, with demand expected to increase as AI applications expand.

What are popular job titles related to Generative Ai Testing jobs in New York?

For Generative Ai Testing jobs in New York, the most frequently searched job titles are:

What job categories do people searching Generative Ai Testing jobs in New York look for?

The top searched job categories for Generative Ai Testing jobs in New York are:

What cities in New York are hiring for Generative Ai Testing jobs?

Cities in New York with the most Generative Ai Testing job openings:

Generative Ai Testing jobs near you
Infographic showing various Generative Ai Testing job openings in New York as of August 2026, with employment types broken down into 1% As Needed, 83% Full Time, 12% Part Time, 3% Contract, and 1% Nights. Highlights an 88% Physical, 3% Hybrid, and 9% Remote job distribution.

Automation Engineer (AI/ML & API Testing)

CYNET SYSTEMS

Jersey City, NJ • On-site

$58 - $67/hr

Contractor

Re-posted 8 days ago

Job description

Job Overview:

Pay Range: $58.52hr - $67.85hr

Requirement/Must Have:

  • 10+ years of proven experience in the software development industry, participating in a team environment.
  • 5+ years of experience in UI automation, such as Selenium WebDriver.
  • 5+ years of proven experience in testing web services such as RESTful API.
  • 3+ years of proven experience in performance testing using tools such as Apache JMeter.
  • 3+ years of working in a cloud-based development and production environment such as AWS.
  • 3+ years of proven experience with programming languages such as Java/C#. Familiarity with Python is a plus.
  • Proficient with both SQL and NoSQL databases.
  • Experienced working in Continuous Integration/Continuous Deployment (CI/CD) environments.
  • Having AWS/Cloud technologies and AI/ML powered applications knowledge are big plus.
  • Experience with AI/ML-powered applications, Generative AI, and Agentic AI systems is highly desirable.
  • Understanding of AI testing concepts, including prompt validation, model output verification, hallucination detection, bias testing, guardrail validation, and AI quality evaluation.
  • Familiarity with LLMs (Large Language Models), AI agents, retrieval-augmented generation (RAG), and AI-assisted workflows is a strong plus.
  • Ability to develop QA strategies and test approaches for AI-driven and autonomous agent-based applications.

Responsibilities:

  • Design and execute functional and automation testing for applications.
  • Develop and execute automated test scripts using testing tools; document and summarize results.
  • Build, update, and maintain UI automation test cases using Selenium with C# (flexibility to learn C# if not already proficient).
  • Perform manual API testing using tools such as Postman or Insomnia.
  • Perform API automation testing using industry-standard frameworks and tools.
  • Participate actively in Agile ceremonies (daily stand-ups, sprint planning, retrospectives).
  • Collaborate closely with Development and Business teams to ensure project success.
  • Quickly adapt to new technologies and tools (e.g., Amazon Web Services).
  • Work with SQL and NoSQL databases for backend validation and robust data testing.
  • Create comprehensive end-to-end test plans, test specifications, and test case templates.
  • Develop and document detailed test plans and test cases.
  • Facilitate defect tracking, reporting, and resolution management.
  • Review requirements (Functional Specifications, User Stories, Change Requests) to ensure full coverage.
  • Identify and manage test data; prepare automation-ready test cases.
  • Conduct system testing, regression testing, and support User Acceptance Testing (UAT).
  • Apply test case design techniques (e.g., boundary value analysis, equivalence partitioning).
  • Implement Shift Left Testing practices by participating in early requirement and design reviews, writing acceptance criteria up front, and integrating automated tests in CI/CD pipelines.
  • Be actively involved in the deployment process, collaborating with DevOps and engineering teams to ensure smooth releases.
  • Design and execute test strategies for AI/ML and Agentic AI applications, including validating AI-generated outputs for accuracy, consistency, and reliability.
  • Test AI agents for workflow execution, decision-making, and multi-step task completion.
  • Evaluate AI model behavior, prompt responses, hallucinations, and edge cases.
  • Verify AI safety controls, guardrails, security, and compliance requirements.
  • Create test datasets and benchmarks for AI quality assessment.
  • Monitor and report AI model performance, accuracy, and user experience metrics.

Nice to Have:

  • Experience with application monitoring tools (e.g., New Relic, Datadog, CloudWatch).
  • Working knowledge of Python for scripting and automation.
  • Familiarity with AWS services for cloud-based testing and deployment.
  • Hands-on experience with performance and load testing using JMeter.
  • Experience with testing AI/ML-powered applications, including validating model outputs, creating test datasets, evaluating model performance.

Founded in 2010 and headquartered in the Washington, DC metro area, Cynet Systems Inc. is a leading staffing and recruiting powerhouse. Proudly recognized as a nationally and locally certified diversity firm, Cynet delivers agile, scalable talent solutions across industries. With an active footprint in all 50 U.S. states and Canada, we support thousands of consultants through our expansive, high-performing recruitment engine operating across North America and Asia—ensuring speed, quality, and consistency in every hire.
Cynet Systems logo

About Cynet Systems

Sourced by ZipRecruiter

Cynet Systems Inc is a staffing and recruiting corporation nestled in Ashburn, VA, USA. Established in 2010, the company operates within the Information Technology and Services sector, specializing in providing effective workforce solutions to different business needs, including IT consulting, direct hire, and contract staffing services. Through the years, Cynet Systems has built an impressive portfolio, going beyond borders and expanding its operations internationally in Canada and India. Rooted in its core values of teamwork, leadership, and commitment, Cynet Systems helps businesses unlock their full potential by providing versatile and competent professionals that perfectly align with their needs. Fueled by their unwavering mission to deliver top-tier talent to businesses worldwide, Cynet Systems garnered various recognitions including SIA's fastest-growing staffing firms and Best Place to Work in Virginia for 2019.

Industry

It services

Company size

501 - 1,000 Employees

Headquarters location

Sterling, VA, US

Year founded

2010

Social media

Twitter

View All Cynet Systems Jobs