ZipRecruiter

Log In

1

Evaluation Research Jobs in San Ramon, CA (NOW HIRING)

Vals

Member of Technical Staff - Research

San Francisco, CA · On-site

$140K - $185K/yr

You will create high-impact, challenging evaluations that push the boundaries of what we can measure in foundation models. This role is ideal for someone with deep research expertise who wants to see ...

Vals AI, Inc.

Head of Research

San Francisco, CA · On-site

$220 - $340/hr

How do you build evaluations that hold up as models improve at gaming them? You will set the ... Oversee Vals' broader research portfolio, setting direction across ongoing projects and new ...

Vals

Head of Research

San Francisco, CA · On-site

$225K - $275K/yr

How do you build evaluations that hold up as models get better at gaming them? You'll be the person ... Recruit and grow a research team alongside the founders. * Work directly with our enterprise ...

Luma

Research Engineer - Evaluations

Redwood City, CA · On-site

$190K - $375K/yr

As a Research Engineer on Evaluations, you'll build the pipelines, metrics, and automated systems ... You'll be turning human evaluation frameworks into automated ones and wiring the results straight ...

next page

Showing results 1-20

All JobsEvaluation Research JobsEvaluation Research Jobs in San Ramon, CA

Evaluation Research information

See San Ramon, CA salary details

$41.3K

$118.5K

$159.2K

How much do evaluation research jobs pay per year?

As of Aug 26, 2026, the average yearly pay for evaluation research in San Ramon, CA is $118,471.00, according to ZipRecruiter salary data. Most workers in this role earn between $116,200.00 and $116,200.00 per year, depending on experience, location, and employer.

What is evaluation research?

Evaluation research is a systematic method of assessing the design, implementation, and outcomes of programs, policies, or projects to determine their effectiveness and inform decision-making. It combines quantitative and qualitative research methods to collect and analyze data, providing evidence-based insights about what works and why. Evaluation research is commonly used in fields such as education, healthcare, social services, and public policy to improve program efficiency and impact.

What are the key skills and qualifications needed to thrive in evaluation research, and why are they important?

To succeed in Evaluation Research, you need a solid background in research methods, data analysis, and program evaluation, often supported by a degree in social sciences or a related field. Proficiency with statistical software such as SPSS, R, or NVivo, and familiarity with survey tools and data collection platforms are typically required. Strong written and verbal communication, critical thinking, and attention to detail help present findings clearly and work effectively with stakeholders. These skills ensure rigorous, actionable evaluations that inform decision-making and improve program outcomes.

What types of projects or sectors do evaluation researchers typically work on, and how does this diversity impact daily responsibilities?

Evaluation researchers often work across a variety of sectors, including education, public health, social services, and nonprofit organizations. This diversity means that your daily tasks can range from designing data collection tools and conducting field interviews to analyzing large datasets and presenting findings to stakeholders. The role requires adaptability, as each project may have different methodologies, objectives, and reporting requirements. Collaborating with subject matter experts and program managers is common, making strong communication and project management skills essential.

What is the difference between Evaluation Research vs Data Analyst?

AspectEvaluation ResearchData Analyst
Required CredentialsMaster's degree in social sciences, research methods, or related fieldsBachelor's or Master's in Data Science, Statistics, or related fields
Work EnvironmentResearch institutions, government agencies, NGOsCorporate, finance, healthcare, tech companies
Employer & Industry UsageUsed in program assessment, policy analysis, social researchUsed in data interpretation, reporting, business insights

Evaluation Research focuses on assessing programs and policies through systematic research, often in social or public sectors. Data Analysts interpret and visualize data to support business decisions. While both roles require analytical skills, Evaluation Research emphasizes research design and evaluation methods, whereas Data Analysts focus on data manipulation and reporting.

What job categories do people searching Evaluation Research jobs in San Ramon, CA look for?

The top searched job categories for Evaluation Research jobs in San Ramon, CA are:

What cities near San Ramon, CA are hiring for Evaluation Research jobs?

Cities near San Ramon, CA with the most Evaluation Research job openings:

Evaluation Research jobs near you
Infographic showing various Evaluation Research job openings in San Ramon, CA as of August 2026, with employment types broken down into 1% As Needed, 83% Full Time, 15% Part Time, and 1% Contract. Highlights an 77% Physical, 4% Hybrid, and 19% Remote job distribution, with an average salary of $118,471 per year, or $57 per hour.

Research Scientist (Model Evaluation)

Sanas

Palo Alto, CA • On-site

Full-time

Re-posted 21 days ago

Job description

Sanas is pioneering the future of human communication. Founded by a team of Stanford researchers and entrepreneurs with deep industry experience, Sanas has developed the world's first real-time speech AI platform capable of accent translation, noise cancellation, speech enhancement, cross-language communication, and more.
Sanas makes conversations clearer, more inclusive, and more effective, removing barriers that prevent people from being understood, regardless of accent, background noise, or native language.
Sanas is currently one of the fastest growing startups in Silicon Valley, growing from $16M to $50M ARR in 2025. The company's core business is profitable and is on track to end 2026 with >$120M ARR. Our team combines deep expertise in model innovation and systems engineering with a design-minded product engineering culture to build and ship cutting-edge AI models and experiences - entirely in-house.
Sanas is a 130 person team, established in 2020. In this short span, we've successfully secured over $100 million in funding. Our innovation has been supported by the industry's leading investors, including Insight Partners, Google Ventures, Quadrille Capital, General Catalyst, Quiet Capital, and other influential investors. Our reputation is further solidified by collaborations with numerous Fortune 100 companies. With Sanas, you're not just adopting a product; you're investing in the future of communication.
If you're looking to have a significant role in roadmapping and driving technical directions, if you're looking to deploy challenging and big ideas without much overhead or slowness, if you're looking to leave your mark on an ambitious, generational mission to change how the worlds thinks about speech + AI, then Sanas is a well-suited place for you.
About the Role
Progress in speech AI is only as meaningful as our ability to measure it. At Sanas, model quality spans dimensions that automated metrics struggle to capture - accent naturalness, perceptual clarity, speaker identity preservation, noise suppression without speech distortion, translation fluency under real-world disfluency. We're looking for a Research Scientist who can define what "better" actually means across all of Sanas's model families, build the evaluation infrastructure to measure it rigorously, and close the loop between research progress and real-world impact. This role sits at the intersection of research, product, and infrastructure - and directly shapes how every model team at Sanas measures progress.
Job Description
Evaluation framework design
  • Design and own evaluation frameworks across Sanas's full model portfolio - Accent Translation, Noise Cancellation, Speech Enhancement, and Language Translation, and more - ensuring each captures meaningful progress, not just benchmark performance.
  • Develop novel quantitative metrics for subjective and perceptual qualities: accent similarity, naturalness, speaker identity preservation, intelligibility under noise, and translation fluency in spoken-language domains.
  • Build evaluation systems that bridge automated metrics and human judgment - designing listening studies, MOS/MUSHRA protocols, and preference tests that are statistically rigorous and operationally scalable.
  • Define evaluation splits, test sets, and benchmark suites that accurately reflect production conditions - diverse accents, languages, noise environments, recording devices, and telephony codecs.

Evaluation infrastructure & tooling
  • Build and maintain automated evaluation pipelines that run continuously against model checkpoints - surfacing regressions early and tracking quality trends across training runs.
  • Develop reference-based and reference-free metrics calibrated to Sanas's specific model tasks: SI-SDR, PESQ, STOI, DNSMOS, speaker similarity, WER delta, COMET, and task-specific custom metrics where off-the-shelf measures fall short.
  • Instrument model quality monitoring in production - detecting degradation across language pairs, accent profiles, and acoustic conditions in live customer traffic.
  • Build tooling that allows research scientists and ML engineers to run rigorous ablations, compare model versions, and understand quality tradeoffs without needing to design the evaluation from scratch each time.

Human evaluation & research
  • Design and operate human evaluation programs - listener panels, crowdsourced annotation, and expert evaluator workflows - that produce reliable signal on dimensions automated metrics cannot capture.
  • Conduct research into evaluation methodology itself: when do automated metrics correlate with human perception, when do they diverge, and what does that tell us about model behavior?
  • Partner directly with research scientists across model teams to translate open-ended quality questions into concrete, measurable evaluation protocols.

Cross-functional impact
  • Work closely with ML research, product, and customer success teams to ensure evaluation reflects what customers actually experience - not just what lab conditions optimize for.
  • Feed evaluation insights back into data acquisition and model training priorities - identifying which failure modes require more data, architectural changes, or training procedure improvements.
  • Communicate evaluation results clearly to both technical and non-technical stakeholders, translating metric movements into product quality narratives that inform roadmap decisions.

Qualifications
  • 4+ years of research or applied research experience in speech, audio, or NLP, with a demonstrated focus on evaluation methodology and quality measurement.
  • Deep familiarity with speech and audio quality metrics - perceptual (MOS, MUSHRA, PESQ, STOI), signal-level (SI-SDR, SNR), and task-specific (WER, speaker similarity, DNSMOS) - and an understanding of when each is and isn't the right tool.
  • Experience designing and running human evaluation studies - listener panels, crowdsourced annotation, inter-annotator agreement analysis - with statistical rigor.
  • Strong engineering skills: you can build production-quality evaluation pipelines, not just run scripts. Proficiency in Python and PyTorch or equivalent.
  • Creativity in defining novel quantitative metrics for subjective or behavioral qualities - you've identified gaps in existing evaluation approaches and built something better.
  • Ability to take open-ended research questions and translate them into concrete, measurable evaluation systems that run reliably at scale.
  • Curiosity and rigor in equal measure - you're as motivated by discovering the right way to measure progress as by the progress itself.

Bonus
  • Experience evaluating models across multiple speech tasks - ASR, TTS, speech enhancement, speaker verification, or machine translation.
  • Familiarity with real-time or streaming model evaluation - latency-quality tradeoffs, codec-degraded audio, telephony channel conditions.
  • Background in psychoacoustics or perceptual audio quality - understanding of how humans perceive speech naturalness, noise, and distortion.
  • Experience with multilingual evaluation - cross-lingual quality metrics, language-specific annotation challenges, low-resource language evaluation.
  • Published research at INTERSPEECH, ICASSP, ACL, EMNLP, or equivalent venues on evaluation methodology, speech quality, or related topics.

Apply