ZipRecruiter

Log In

1

Generative Ai Testing Jobs in New York (NOW HIRING)

Uncountable

Platform Engineer - Generative AI

New York, NY · On-site

$120K - $160K/yr

... tuning/testing behavior) and traditional fullstack development tasks (coding frontends/backends ... Strong interest in Generative AI * Solid computer science and software engineering fundamentals ...

Hearst

Senior Engineer, Generative AI

New York, NY

$114K - $157K/yr

Design and develop Generative AI applications and features using modern LLM APIs and open-source ... A commitment to automated testing and software development best practices * Strong problem solving ...

Jj

Data and Generative AI Engineer

Raritan, NJ

$117K - $140K/yr

Design, develop and deliver Generative/Agentic AI solutions that accelerate End-to-End Data ... Ensure data quality and integrity through meticulous testing and validation, applying Intelligent ...

Jj

Data and Generative AI Engineer

New Brunswick, NJ

$118K - $141K/yr

Design, develop and deliver Generative/Agentic AI solutions that accelerate End-to-End Data ... Ensure data quality and integrity through meticulous testing and validation, applying Intelligent ...

next page

Showing results 1-20

All JobsGenerative Ai Testing JobsGenerative Ai Testing Jobs in New York

Generative Ai Testing information

What is the difference between Generative Ai Testing vs Data Scientist?

AspectGenerative Ai TestingData Scientist
Required CredentialsKnowledge of AI models, testing tools, programming skillsStatistics, programming, data analysis certifications
Work EnvironmentAI development teams, testing labs, tech companiesResearch labs, tech firms, finance, healthcare
Employer & Industry UsageAI product testing, quality assurance in techData analysis, predictive modeling across industries

Generative Ai Testing focuses on evaluating and validating AI-generated content and models, ensuring quality and accuracy. Data Scientists analyze data, build models, and derive insights. While both roles require programming and AI knowledge, Generative Ai Testing emphasizes testing processes, whereas Data Scientists focus on data analysis and model development.

How much do AI testers get paid?

AI testers, involved in evaluating and validating generative AI models, typically earn salaries ranging from $60,000 to $120,000 annually depending on experience, location, and company size. Entry-level positions may start lower, while experienced testers with specialized skills in machine learning and data analysis can earn higher wages.

Is AI testing a good career?

AI testing, including roles like Generative AI Testing, is a growing field with increasing demand for skills in machine learning, data analysis, and software quality assurance. It offers opportunities in tech companies, research labs, and startups, often requiring knowledge of AI frameworks and testing tools. The career can be stable and rewarding for those with technical expertise and an interest in AI development.

What are the key skills and qualifications needed to thrive as a Generative AI Testing Specialist, and why are they important?

To thrive as a Generative AI Testing Specialist, you need a robust understanding of machine learning principles, model evaluation techniques, and a background in computer science or a related field. Familiarity with tools such as Python, TensorFlow, PyTorch, and model evaluation frameworks, as well as experience with automated testing platforms, is typically required. Analytical thinking, attention to detail, and strong communication skills help you identify model weaknesses and collaborate effectively with development teams. These skills are crucial to ensure the reliability, safety, and ethical deployment of generative AI solutions.

What is the salary of generative AI tester?

The salary of a generative AI tester typically ranges from $70,000 to $120,000 annually, depending on experience, location, and company size. Entry-level positions may start lower, while experienced testers with specialized skills in AI and machine learning can earn higher salaries. Certifications in AI or related fields can also influence compensation.

What are some common challenges faced when testing generative AI models, and how can I prepare to address them in this role?

Testing generative AI models often involves unique challenges such as evaluating the quality and relevance of generated content, detecting bias or inappropriate outputs, and ensuring model consistency across various prompts. You may work closely with data scientists and engineers to create robust evaluation frameworks and develop automated as well as manual testing strategies. Familiarity with prompt engineering, statistical evaluation techniques, and domain-specific knowledge will help you address these challenges effectively. Proactively staying updated on industry best practices and collaborating with cross-functional teams are key to success in this dynamic field.

How do I become an AI tester?

To become an AI tester, you should have a strong understanding of machine learning concepts, programming skills in languages like Python, and experience with data annotation and model evaluation. Familiarity with AI tools, testing frameworks, and quality assurance processes is also important. Gaining relevant certifications or training in AI and software testing can enhance your qualifications.

What is Generative AI Testing?

Generative AI Testing refers to the process of evaluating and validating AI systems, particularly those that generate content such as text, images, or code. This type of testing focuses on assessing the accuracy, reliability, fairness, and safety of generative models to ensure they function as intended and avoid producing harmful or biased outputs. Testers use various methods, including automated and manual techniques, to check for issues like hallucinations, inappropriate content, or security vulnerabilities. The goal is to build trust in generative AI systems and ensure they meet quality and ethical standards before deployment.
What are popular job titles related to Generative Ai Testing jobs in New York? For Generative Ai Testing jobs in New York, the most frequently searched job titles are:
What job categories do people searching Generative Ai Testing jobs in New York look for? The top searched job categories for Generative Ai Testing jobs in New York are:
What cities in New York are hiring for Generative Ai Testing jobs? Cities in New York with the most Generative Ai Testing job openings:
Generative Ai Testing jobs near you
Infographic showing various Generative Ai Testing job openings in New York as of July 2026, with employment types broken down into 75% Full Time, 22% Part Time, and 3% Contract. Highlights an 71% Physical, 3% Hybrid, and 26% Remote job distribution.
Hiring QA Gen AI Automation Engineer for Jersey City, NJ

Hiring QA Gen AI Automation Engineer for Jersey City, NJ

Hexacorp

Jersey City, NJ • On-site

Other

Posted 5 days ago

Job description

Hiring QA Gen AI Automation Engineer for Jersey City, NJ

Role: QA Gen AI Automation Engineer
Location: Jersey City, NJ
Duration: Long Term

Key Responsibilities:
Automation Testing:
Develop and maintain automated regression tests using tools like Selenium WebDriver/Playwright and pytest.
Conduct API testing using tools such as Postman.
Generative AI Testing:
Design and execute test cases for Generative AI models and applications.
Evaluate the performance and quality of Generative AI outputs, ensuring they meet functional requirements and quality standards.
Validate AI systems such as Chatbots, RAG applications, and LLM-based workflows.
Perform LLM evaluation using frameworks such as DeepEval and RAGAS.
Programming & Scripting:
Write and maintain automated test scripts in Python.
Work with XML, JSON, and other scripting languages to create efficient testing environments for AI systems.
Testing Tools & Frameworks:
Utilize JMeter for performance testing and load testing.
Implement and manage Continuous Integration (CI) pipelines using tools such as Jenkins and Bamboo.
Database & SQL:
Hands on experience with Panda Data frames
Execute SQL queries for database testing (inserts, updates, joins, etc.).
Test non-GUI applications (e.g., SQL, flat files, XML).
Work with Vector Databases such as OpenSearch or similar vector DB technologies.
Agile & DevOps:
Collaborate within an Agile (Scrum) framework for efficient testing and development.
Utilize Atlassian JIRA and Confluence for project tracking and documentation.
Product Evaluation & Troubleshooting:
Identify and troubleshoot product issues, ensuring quality and stability.
Evaluate product performance and provide feedback to development teams.
Web Application Testing:
Conduct testing for web applications and web services (REST, SOAP).
Required Skills & Experience:
Programming:
Proficient in Python for writing test scripts and automating processes.
Automation Testing:
Experience with Selenium WebDriver/Playwright and pytest for automation testing.
Hands-on experience in API testing using tools like Postman.
Generative AI & AI Testing:
Understanding of AI systems and AI Testing concepts.
Knowledge of Large Language Models (LLMs), RAG, and Chatbot AI systems.
Experience designing and testing Generative AI systems and applications.
Knowledge of LLM evaluation techniques and frameworks such as DeepEval and RAGAS.
Ability to assess the output quality, coherence, and accuracy of Generative AI models.
Database & Search Technologies:
Strong understanding of SQL for testing databases (including joins, inserts, and updates).
Understanding of Vector Databases such as OpenSearch or similar vector DB technologies.
Testing Tools:
Experience with JMeter for performance and load testing.
Familiarity with CI/CD tools like Jenkins or Bamboo.
Agile Methodology:
Experience working in an Agile (Scrum) environment.
Familiar with JIRA and Confluence for issue tracking and project management.
Web & API Testing:
Experience testing web applications and web services.
Product Performance:
Capable of evaluating and troubleshooting performance-related issues.
Nice to Have:
Agentic AI & Observability:
Understanding of Agents, MCP, and Agentic AI systems.
Familiarity with observability tools such as Langfuse.
Cloud Experience:
Familiarity with AWS services such as AWS Bedrock for cloud-based AI environments.
Development Experience:
A background in software development is a plus.
Industry Knowledge:
Understanding of the Property and Casualty (P&C) insurance industry is beneficial.

Apply