Llm Evaluation Jobs in Bothell, WA (NOW HIRING)

... LLM evaluation science for AI-powered agent experiences, or agent memory architectures that enable persistent, adaptive behavior across sessions. You will work closely with applied scientists ...

... LLM evaluation science for AI-powered agent experiences, or agent memory architectures that enable persistent, adaptive behavior across sessions. You will work closely with applied scientists ...

Conduct LLM evaluation using automated and human-in-the-loop techniques (offline + online). * Optimize inference workflows for latency , GPU utilization , and cost efficiency (quantization, batching ...

... evaluation, retraining, versioning, and observability of ML/LLM systems • Drive adoption through documentation, training, and internal evangelism • Collaborate with engineering to level up Agoda ...

Worked with LLM evaluation, observability, or guardrails * Have experience with cloud environments (AWS, GCP, Azure), containers, and basic Kubernetes concepts * Have shipped and operated production ...

Design rigorous evaluation frameworks to measure model performance, bias, safety, and business ... LLM applications (prompt engineering, RAG, fine-tuning, RLHF, Transformer models, and AI Agents ...

Define's Client enterprise AI strategy and architecture, including platform selection, LLM evaluation, governance model, technical standards, and alignment with Client's long-term digital and ...

Define's Client enterprise AI strategy and architecture, including platform selection, LLM evaluation, governance model, technical standards, and alignment with Client's long-term digital and ...

Experience with AI safety, LLM evaluation, prompt engineering, or red teaming. * Background in trust & safety, content moderation, or behavioral policy development. * Experience developing evaluation ...

Build LLM evaluation pipelines covering relevance, groundedness, and hallucination metrics integrated into CI/CD workflows to enforce automated quality gates before every model or agent deployment.

... LLM evaluation, and fine-tuning, to develop production-ready applications powered by foundation models - Developing automated evaluation frameworks, including LLM-as-judge pipelines, regression ...

Experience with AI safety, LLM evaluation, prompt engineering, or red teaming. * Background in trust & safety, content moderation, or behavioral policy development. * Experience developing evaluation ...

Experience with AI safety, LLM evaluation, prompt engineering, or red teaming. * Background in trust & safety, content moderation, or behavioral policy development. * Experience developing evaluation ...

Driving hands-on execution across human evaluation, and LLM-based autograders and rubrics, and ensuring those methods translate into measurable model and agentic feature improvements. Partnering with ...

LLM-as-judge calibration, rubric design, and bias detection; intelligent evaluation strategies including active learning for test selection and automated failure discovery; or validity frameworks for ...

Successful candidates will have strong experience in traditional human evaluation methodology, in addition to hands-on experience building and deploying LLM-based autograders and rubrics, and using ...

Experience with AI safety, LLM evaluation, prompt engineering, or red teaming. * Background in trust & safety, content moderation, or behavioral policy development. * Experience developing evaluation ...

Driving hands-on execution across human evaluation, and LLM-based autograders and rubrics, and ensuring those methods translate into measurable model and agentic feature improvements. * Partnering ...

... LLM evaluation science for AI-powered agent experiences, or agent memory architectures that enable persistent, adaptive behavior across sessions. You will work closely with applied scientists ...

Experience with AI safety, LLM evaluation, human-in-the-loop systems, annotation, red teaming, or trust & safety. * Experience leading multi-site operations or facilities coordination. * Experience ...