ZipRecruiter

Log In

1

Map Evaluation Job Jobs (NOW HIRING)

LTS

RAG and Evaluation Engineer

OR · On-site +1

Benchmarks for translation accuracy, dependency-map correctness, and overall agent quality. The ... Strong applied evaluation skills - benchmark design, regression detection, LLM-as-judge patterns.

LTS

RAG and Evaluation Engineer

Benchmarks for translation accuracy, dependency-map correctness, and overall agent quality. The ... Strong applied evaluation skills - benchmark design, regression detection, LLM-as-judge patterns.

Operation Blessing

Monitoring & Evaluation Coordinator

$52K - $56K/yr

Based) Operation Blessing (OB) is seeking a detail-oriented Monitoring & Evaluation Coordinator to ... Experience with GIS mapping and spatial data visualization tools * Self-motivated with the ability ...

next page

Showing results 1-20

All JobsMap Evaluation Job Jobs

Map Evaluation Job information

What is the difference between Map Evaluation Job vs GIS Analyst?

AspectMap Evaluation JobGIS Analyst
Required CredentialsGIS certification, cartography skillsGIS certification, spatial analysis skills
Work EnvironmentField and office, map review and validationOffice-based, data analysis and mapping
Employer & Industry UsageUrban planning, environmental agenciesGovernment, transportation, environmental firms
Search & Comparison IntentUnderstanding map validation rolesAnalyzing spatial data and creating maps

The Map Evaluation Job primarily focuses on reviewing and validating maps, often requiring cartography and GIS certifications. In contrast, a GIS Analyst performs spatial data analysis and creates detailed maps. While both roles involve GIS skills, the Map Evaluation Job emphasizes map accuracy and quality control, whereas the GIS Analyst emphasizes data analysis and visualization.

What job categories do people searching Map Evaluation Job jobs look for? The top searched job categories for Map Evaluation Job jobs are:
Map Evaluation Job jobs near you
Infographic showing various Map Evaluation Job job openings in the United States as of June 2026, with employment types broken down into 1% As Needed, 3% Full Time, 95% Part Time, and 1% Contract. Highlights an 92% Physical, 2% Hybrid, and 6% Remote job distribution.
RAG and Evaluation Engineer

RAG and Evaluation Engineer

LTS

OR • On-site, Remote

Other

Medical

Posted 18 days ago

Key responsibilities

  • Own ingestion pipelines for source code, structured metadata, technical documentation, patches, and additional corpora provided by the customer.

  • Own retrieval quality across chunking, embeddings, hybrid retrieval, reranking, and freshness.

  • Own the evaluation harness including benchmarks for translation accuracy, dependency-map correctness, and overall agent quality.

Job description

LTS is seeking a RAG & Evaluation Engineer to join a small, senior engineering team applying frontier AI to one of the most consequential legacy systems still running in production today.

The mission: build agents that read, translate, and modernize a decades-old codebase that millions of people quietly depend on. The work has executive backing, real users, and a customer who knows exactly what they're buying. Specifics shared once we're talking.

The team is small by design. Every seat carries unusual leverage, and we hire people who are already deep in this work. We use AI tooling natively - agents in parallel, model as collaborator, no exceptions.

What You'll Do:

The RAG & Evaluation Engineer owns the knowledge surface and the eval harness. Ingestion pipelines for source code, structured metadata, technical documentation, patches, and additional corpora the customer provides. Retrieval quality across chunking, embeddings, hybrid retrieval, reranking, freshness. Benchmarks for translation accuracy, dependency-map correctness, and overall agent quality. The feedback loop from production usage back into evals and retrieval lives here.

  • Own the knowledge surface - ingestion pipelines for source code, structured metadata, technical documentation, patches, and additional corpora the customer provides.
  • Own retrieval quality - chunking, embeddings, hybrid retrieval, reranking, and freshness.
  • Own the eval harness - benchmarks for translation accuracy, dependency-map correctness, and overall agent quality.
  • Run A/B testing and regression detection across prompts, retrieval, and model changes.
  • Operate the feedback loop from production usage back into evals and retrieval.
  • Define what "good" means for the platform when no one else has a clear view, so the team can tell whether the agent is actually improving.
  • Pair with the Agent Engineers on the prompt-and-eval iteration cycle.

What We're Looking For:

  • Bachelor's degree in Computer Science, Engineering, Information Science, or a related field, plus 4 years of professional software engineering experience; equivalent experience may substitute for the degree requirement.
  • Has shipped a production RAG system with quality the candidate can describe in numbers (rigor matters more than scale).
  • Ability to work in a fast-paced, collaborative environment.
  • Production experience with retrieval pipelines - ingestion, chunking, embedding, hybrid retrieval, reranking.
  • Strong applied evaluation skills - benchmark design, regression detection, LLM-as-judge patterns.
  • Knows when BM25 beats embeddings and when neither is enough.
  • Measures everything they ship; opinions about chunking are backed by benchmarks.
  • Patient with detail; comfortable defining metrics before the team has agreed on them.
  • Heavy native use of AI tooling: agents in parallel, model as collaborator.
  • Strong TypeScript or Python.
  • Demonstrated experience in a remote work environment.

Nice to Have:

  • Code-as-corpus retrieval (search over source code rather than prose).
  • Applied IR or search-engine background.
  • Synthetic data generation and LLM-as-judge patterns.
  • Open-source contributions to retrieval, eval, or RAG tooling.
  • Experience integrating retrieval feedback loops with production usage.
  • Healthcare IT or legacy modernization domain experience.
  • Public technical writing or conference talks on retrieval or evaluation.

What's in it for you? 

  • The opportunity to support high visibility federal missions in IT and healthcare
  • A culture that values innovation, growth, collaboration, and quality
  • Access to cutting-edge tools and technologies
  • Comprehensive benefits for you and your family
  • A career path that rewards ambition and performance

If you're ready to push boundaries, sharpen your skills, and join a team that is passionate about building what's next, we'd love to meet you. Apply today and let's build a future together! 

Apply