ZipRecruiter

Log In

1

Large Language Model Tester Jobs (NOW HIRING)

Peraton

Large Language Model Specialist

$104K - $166K/yr

... large-language-model capabilities across a cloud-based analytics ecosystem. This role leads ... Solid understanding of MLOps/LLMOps practices, including versioning, CI/CD, automated testing ...

CACI International Inc.

Principal AI/ML Engineer (Large Language Model) Job Category: Science Time Type: Full time Minimum Clearance Required to Start: TS/SCI with Polygraph Employee Type: Regular Percentage of Travel ...

next page

Showing results 1-20

All JobsTechnology JobsSoftware JobsLarge Language Model Tester Jobs

Large Language Model Tester information

See salary details

$10

$38

$62

How much do large language model tester jobs pay per hour?

As of Sep 9, 2026, the average hourly pay for large language model tester in the United States is $38.36, according to ZipRecruiter salary data. Most workers in this role earn between $21.39 and $50.72 per hour, depending on experience, location, and employer.

What are popular job titles related to Large Language Model Tester jobs?

For Large Language Model Tester jobs, the most frequently searched job titles are:

Infographic showing various Large Language Model Tester job openings in the United States as of August 2026, with employment types broken down into 90% Full Time, 7% Part Time, and 3% Contract. Highlights an 90% Physical, 3% Hybrid, and 7% Remote job distribution, with an average salary of $79,791 per year, or $38.4 per hour.

Large Language Model Specialist

Remote

Peraton
IT Services • 10K+ employees
Learn more about Peratonexternal

$104K - $166K/yr

Full-time

Re-posted 6 days ago

Peraton rating

8.3

Company rating: 8.3 out of 10

Based on 56 frontline employees who took The Breakroom Quiz

52nd of 226 rated it services

Job description

Responsibilities
The LLM Specialist will drive the design, development, and operationalization of advanced large-language-model capabilities across a cloud-based analytics ecosystem. This role leads innovation efforts around cutting-edge AI, owning the architecture and strategy for fine-tuning, retrieval-augmented generation (RAG), agentic frameworks, and domain-specific model adaptation. The specialist will guide the development of high-impact prototypes, oversee the evolution of scalable LLM pipelines, and ensure robust governance, security, and performance across all model implementations. Partnering with engineering, product, and data teams, this position provides technical leadership, evaluates emerging LLM technologies, sets best practices, and helps drive transformation through the practical, safe, and effective deployment of generative AI.
Qualifications
Minimum of 8 years with BS/BA; Minimum of 6 years with MS/MA; Minimum of 3 years with PhD
Required Skills:
  • Deep expertise in LLM architectures, transformer models, and modern generative AI techniques.
  • Demonstrated experience leading fine-tuning efforts, parameter-efficient training, and advanced prompt engineering.
  • Proven ability to design and implement end-to-end RAG pipelines, including embedding workflows, retrieval optimization, and vector database integrations.
  • Hands-on experience with one or more LLM frameworks or orchestration toolchains (such as LangChain, LlamaIndex).
  • Strong Python development skills and experience with distributed compute or GPU-accelerated training environments.
  • Experience architecting and deploying AI/ML or LLM workflows within cloud platforms such as Azure, AWS, or GCP.
  • Solid understanding of MLOps/LLMOps practices, including versioning, CI/CD, automated testing, monitoring, and model governance.
  • Ability to lead technical discussions, mentor team members, and communicate complex AI concepts to diverse audiences.
  • Ability to obtain/maintain a Public Trust clearance

Preferred Skills:
  • Experience implementing multi-agent or agentic AI systems for task automation and reasoning.
  • Familiarity with LLM evaluation frameworks, structured benchmarking, or human-in-the-loop refinement methods (e.g., RLHF-style workflows).
  • Expertise with advanced retrieval techniques such as hybrid search, graph retrieval, or long-context optimization.
  • Experience optimizing model inference through quantization, model compression, or model distillation.
  • Background integrating LLM services with large-scale analytics environments (e.g., Databricks, Snowflake, Spark).
  • Strong skills in exploratory data analysis, feature engineering, and data modeling to support domain-specific LLM customization.
  • Experience developing innovative prototypes or POCs that leverage state-of-the-art generative AI approaches.
  • Exposure to emerging architectures such as mixture-of-experts models, long-context transformers, or experimental generative frameworks.

Peraton Overview
Peraton is a next-generation national security company that drives missions of consequence spanning the globe and extending to the farthest reaches of the galaxy. As the world's leading mission capability integrator and transformative enterprise IT provider, we deliver trusted, highly differentiated solutions and technologies to protect our nation and allies. Peraton operates at the critical nexus between traditional and nontraditional threats across all domains: land, sea, space, air, and cyberspace. The company serves as a valued partner to essential government agencies and supports every branch of the U.S. armed forces. Each day, our employees do the can't be done by solving the most daunting challenges facing our customers. Visit peraton.com to learn how we're keeping people around the world safe and secure.
Target Salary Range
$104,000 - $166,000. This represents the typical salary range for this position. Salary is determined by various factors, including but not limited to, the scope and responsibilities of the position, the individual's experience, education, knowledge, skills, and competencies, as well as geographic location and business and contract considerations. Depending on the position, employees may be eligible for overtime, shift differential, and a discretionary bonus in addition to base pay.
EEO
EEO: Equal opportunity employer, including disability and protected veterans, or other characteristics protected by law.

What Peraton employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom

Peraton logo

About Peraton

Sourced by ZipRecruiter

At Peraton, we re at the forefront of delivering the next big thing every day. We re the partner of choice to help solve some of the world s most daunting challenges, delivering bold, new solutions to keep people around the world safer and more secure.

Industry

It services

Company size

10,000+ Employees

Headquarters location

Herndon, VA, US

Year founded

2017

View All Peraton Jobs

Apply