Llm Evaluation Jobs (NOW HIRING)

Monitoring and evaluation tools like LangSmith, LLM deployment libraries like LiteLLM, LLM Guardrails, and more, CI/CD, model versioning, and model deployment best practices • Knowledge of or ...

... and LLM evaluation workflows • Familiarity with enterprise AI tools such as Claude, GPT-4/4o, ChatGPT Enterprise, Copilot, Glean, and Shelf • Ingest and process unstructured/structured data ...

AI/ML Engineering & MLOps Experience developing, evaluating, deploying, and monitoring ML/LLM solutions, including LLM evaluation metrics, hallucination reduction, safety guardrails, CI/CD, cloud ...

... LLM evaluation Familiarity with MLOps tools and practices, including Monitoring and evaluation tools like LangSmith LLM deployment libraries like LiteLLM, LLM Guardrails, and more CICD, model ...

LLM evaluation, guardrails, privacy, bias, and safety * Scalable, low-latency, multi-tenant distributed systems design * Observability, SLOs, incident response, capacity, and cost management

LLM evaluation, guardrails, privacy, bias, and safety * Scalable, low-latency, multi-tenant distributed systems design * Observability, SLOs, incident response, capacity, and cost management

Data Scientist, Onsite We have an immediate need for an experienced Data Scientist with strong expertise in Generative AI, LLM evaluation, conversational AI, statistical analysis, and machine ...

Familiarity with LLM evaluation metrics and prompt tuning techniques * Experience validating models integrated with Epic or EHR apps

LLM evaluation, guardrails, privacy, bias, and safety * Scalable, low-latency, multi-tenant distributed systems design * Observability, SLOs, incident response, capacity, and cost management

Qualifications * 3+ years of hands-on experience in LLM / GenAI data evaluation. * Bachelor's Degree required * Ability to research unfamiliar topics using trusted sources and make well-supported ...

Qualifications * 3+ years of hands-on experience in LLM / GenAI data evaluation. * Bachelor's Degree required * Ability to research unfamiliar topics using trusted sources and make well-supported ...

LLM evaluation, guardrails, privacy, bias, and safety * Scalable, low-latency, multi-tenant distributed systems design * Observability, SLOs, incident response, capacity, and cost management

SFT data curation, preference data collection for RLHF/DPO, golden datasets, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval (text, image ...

... evaluation frameworks, including datasets and benchmarks, to ensure our LLM applications meet quality standards and perform reliably over time. • Provide mentorship to team members, sharing your ...

As an LLM Ops Engineer, you will create the secure, scalable, and reliable foundation that allows ... Lead the evolution of Litera's AI operations capabilities by evaluating emerging technologies and ...

As an LLM Ops Engineer, you will create the secure, scalable, and reliable foundation that allows ... Lead the evolution of Litera's AI operations capabilities by evaluating emerging technologies and ...

Qualifications * 3+ years of hands-on experience in LLM / GenAI data evaluation. * Bachelor's Degree required * Ability to research unfamiliar topics using trusted sources and make well-supported ...

End-to-end experience with ML pipelines, model versioning, feature stores, drift detection, CI/CD for ML, and Docker containerization • LLM Evaluation: Experience with evaluation frameworks (RAGAS ...

Develop LLM- and ML-assisted workflows to generate high-quality training and evaluation datasets across targeted behaviors, domains, and edge cases * Build automated evaluators, quality checks, and ...

Implement LLM evaluation, observability, guardrails, content filtering, and responsible AI practices. * Troubleshoot, optimize, and support production AI applications. Required Qualifications * 7+ ...