Llm Evaluation Jobs (NOW HIRING)

Own LLM evaluation processes and methods with a focus on generating benchmarks representative of real-world usage and safety vulnerabilities. * Generate high quality synthetic data, curate labels ...

LLM Evaluations Engineering Lead - SF Bay Area (Onsite) Full-time / Permanent We're partnering with a deep‑tech AI company building autonomous, agentic systems for complex physical and real‑world ...

As an Applied Research Scientist, LLM Evaluation & Post-Training, you will lead research and experimentation on how evaluation design, measurement strategies, and feedback signals influence model ...

Design and implement automated systems and pipelines for evaluating LLM outputs. * Develop metrics and KPIs to measure output quality, accuracy, and consistency using LLM-based evaluations

As an Applied Research Scientist, LLM Evaluation & Post-Training, you will lead research and experimentation on how evaluation design, measurement strategies, and feedback signals influence model ...

A leading AI technology firm in Redwood City is seeking an LLM Evaluations Engineering Lead. In this full-time position, you will be responsible for building evaluation systems for agentic LLMs ...

Evaluate LLM outputs for code generation, refactoring, debugging, and implementation tasks. * Identify and document model failures, edge cases, and reasoning gaps. * Perform head-to-head evaluations ...

Evaluate LLM outputs for code generation, refactoring, debugging, and implementation tasks. * Identify and document model failures, edge cases, and reasoning gaps. * Perform head-to-head evaluations ...

Evaluate LLM outputs for code generation, refactoring, debugging, and implementation tasks. * Identify and document model failures, edge cases, and reasoning gaps. * Perform head-to-head evaluations ...

Evaluate LLM outputs for code generation, refactoring, debugging, and implementation tasks. * Identify and document model failures, edge cases, and reasoning gaps. * Perform head-to-head evaluations ...

Evaluate LLM outputs for code generation, refactoring, debugging, and implementation tasks. * Identify and document model failures, edge cases, and reasoning gaps. * Perform head-to-head evaluations ...

Your primary focus will be evaluating production LLM pipelines: designing accuracy metrics, measuring retrieval effectiveness, and making principled cost/quality tradeoffs for the prompts and models ...

Your primary focus will be evaluating production LLM pipelines: designing accuracy metrics, measuring retrieval effectiveness, and making principled cost/quality tradeoffs for the prompts and models ...

Create robust and scalable LLM evaluation frameworks to measure model performance, guide iteration, and prevent regression via CI/CD. * Deploy RAG systems and MCP servers to more effectively ground ...

We are also seeking a Contractor based LLM Evaluation & QA Engineer to support the testing and validation of large language model (LLM)-powered applications. You will help implement test strategies ...

We are also seeking a Contractor based LLM Evaluation & QA Engineer to support the testing and validation of large language model (LLM)-powered applications. You will help implement test strategies ...

Create robust and scalable LLM evaluation frameworks to measure model performance, guide iteration, and prevent regression via CI/CD. * Deploy RAG systems and MCP servers to more effectively ground ...

We are also seeking a Contractor based LLM Evaluation & QA Engineer to support the testing and validation of large language model (LLM)-powered applications. You will help implement test strategies ...

We develop LLM-as-judge evaluators, train reward models calibrated against human feedback, optimize prompts and context for agents, and contribute targeted datasets and reward signals to foundation ...

Implement observability and telemetry for LLM-powered applications, agents, tools, and platform services. Build evaluation suites for agent behavior, prompt quality, response quality, retrieval ...