Multimodal generative systems require: * benchmarking across visual realism, pose consistency, and identity preservation, * automated regression detection across model checkpoints, * scalable ...
Multimodal generative systems require: * benchmarking across visual realism, pose consistency, and identity preservation, * automated regression detection across model checkpoints, * scalable ...
Performance Benchmarking Engineer
$105K - $260K/yr
Knowledge of competitive benchmarking workloads across Intel, AMD & ARM64 platforms * Real world experience with GNU/LLVM tool chain & debuggers like GDB * Capable of working independently with ...
Performance Benchmarking Engineer
$105K - $260K/yr
Knowledge of competitive benchmarking workloads across Intel, AMD & ARM64 platforms * Real world experience with GNU/LLVM tool chain & debuggers like GDB * Capable of working independently with ...
Principal Engineer - Perf and Benchmarking
Bellevue, WA · On-site
$206K - $333K/yr
Strategy & Leadership - Define the multi-year benchmarking strategy and roadmap; prioritize models/workloads (LLMs, diffusion, vision, speech) and hardware tiers. Build, lead, and mentor a high ...
Principal Engineer - Perf and Benchmarking
Bellevue, WA · On-site
$206K - $333K/yr
Strategy & Leadership - Define the multi-year benchmarking strategy and roadmap; prioritize models/workloads (LLMs, diffusion, vision, speech) and hardware tiers. Build, lead, and mentor a high ...
The Operational Performance and Benchmarking Analytics Coordinator (OPBAC) role encompasses all activities required to measure our organization's performance relative to similar organizations in our ...
The Operational Performance and Benchmarking Analytics Coordinator (OPBAC) role encompasses all activities required to measure our organization's performance relative to similar organizations in our ...
Working at the interface of quantum algorithms, architecture and benchmarking, you will analyze how hardware constraints and effective error models impact end-to-end computational performance. You ...
Working at the interface of quantum algorithms, architecture and benchmarking, you will analyze how hardware constraints and effective error models impact end-to-end computational performance. You ...
Principal Engineer - Perf and Benchmarking
$206K - $333K/yr
Strategy & Leadership - Define the multi-year benchmarking strategy and roadmap; prioritize models/workloads (LLMs, diffusion, vision, speech) and hardware tiers. Build, lead, and mentor a high ...
Quick apply
Principal Engineer - Perf and Benchmarking
$206K - $333K/yr
Strategy & Leadership - Define the multi-year benchmarking strategy and roadmap; prioritize models/workloads (LLMs, diffusion, vision, speech) and hardware tiers. Build, lead, and mentor a high ...
Senior Software Engineer - Perf and Benchmarking
$182K - $242K/yr
Develop and enhance Kubernetes-native benchmarking services that measure latency, throughput, jitter, and cost-per-request across CoreWeave's compute stack. * Contribute to implementing and ...
Quick apply
Senior Software Engineer - Perf and Benchmarking
$182K - $242K/yr
Develop and enhance Kubernetes-native benchmarking services that measure latency, throughput, jitter, and cost-per-request across CoreWeave's compute stack. * Contribute to implementing and ...
Performance Benchmarking Engineer
Cupertino, CA · On-site +1
$105K - $260K/yr
Knowledge of competitive benchmarking workloads across Intel, AMD & ARM64 platforms * Real world experience with GNU/LLVM tool chain & debuggers like GDB * Capable of working independently with ...
Performance Benchmarking Engineer
Cupertino, CA · On-site +1
$105K - $260K/yr
Knowledge of competitive benchmarking workloads across Intel, AMD & ARM64 platforms * Real world experience with GNU/LLVM tool chain & debuggers like GDB * Capable of working independently with ...
Knowledge of competitive benchmarking workloads across Intel, AMD & ARM64 platforms * Real world experience with GNU/LLVM tool chain & debuggers like GDB * Capable of working independently with ...
Quick apply
Knowledge of competitive benchmarking workloads across Intel, AMD & ARM64 platforms * Real world experience with GNU/LLVM tool chain & debuggers like GDB * Capable of working independently with ...
Working at the interface of quantum algorithms, architecture and benchmarking, you will analyze how hardware constraints and effective error models impact end-to-end computational performance. You ...
Working at the interface of quantum algorithms, architecture and benchmarking, you will analyze how hardware constraints and effective error models impact end-to-end computational performance. You ...
Performance Benchmarking Engineer
$105K - $260K/yr
Knowledge of competitive benchmarking workloads across Intel, AMD & ARM64 platforms * Real world experience with GNU/LLVM tool chain & debuggers like GDB * Capable of working independently with ...
Performance Benchmarking Engineer
$105K - $260K/yr
Knowledge of competitive benchmarking workloads across Intel, AMD & ARM64 platforms * Real world experience with GNU/LLVM tool chain & debuggers like GDB * Capable of working independently with ...
We are seeking a skilled HPC/AI Benchmarking and Telemetry Engineer to join our team and drive performance insights across our most advanced computing infrastructure. In this role, you'll develop and ...
We are seeking a skilled HPC/AI Benchmarking and Telemetry Engineer to join our team and drive performance insights across our most advanced computing infrastructure. In this role, you'll develop and ...
Senior Software Engineer - Perf and Benchmarking
Bellevue, WA · On-site +1
$182K - $242K/yr
Senior Software Engineer - Perf and Benchmarking Sunnyvale, CA / Bellevue, WA CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology ...
Senior Software Engineer - Perf and Benchmarking
Bellevue, WA · On-site +1
$182K - $242K/yr
Senior Software Engineer - Perf and Benchmarking Sunnyvale, CA / Bellevue, WA CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology ...
Sr. Software Engineer - Perf and Benchmarking
Sunnyvale, CA · On-site
$139K - $204K/yr
Implement and maintain benchmarking workflows for end-to-end MLPerf Training and Inference runs, including workload setup, cluster configuration, runbooks, and result validation. * Lead design ...
Quick apply
Sr. Software Engineer - Perf and Benchmarking
Sunnyvale, CA · On-site
$139K - $204K/yr
Implement and maintain benchmarking workflows for end-to-end MLPerf Training and Inference runs, including workload setup, cluster configuration, runbooks, and result validation. * Lead design ...
We are seeking a skilled HPC/AI Benchmarking and Telemetry Engineer to join our team and drive performance insights across our most advanced computing infrastructure. In this role, you'll develop and ...
We are seeking a skilled HPC/AI Benchmarking and Telemetry Engineer to join our team and drive performance insights across our most advanced computing infrastructure. In this role, you'll develop and ...
As a Machine Learning Applications & Benchmarking Intern, your role will focus on benchmarking machine learning models on various hardware, creating clean code demos, building end-to-end applications ...
As a Machine Learning Applications & Benchmarking Intern, your role will focus on benchmarking machine learning models on various hardware, creating clean code demos, building end-to-end applications ...
Senior Software Engineer - Perf and Benchmarking
Bellevue, WA · On-site
$182K - $242K/yr
Develop and enhance Kubernetes-native benchmarking services that measure latency, throughput, jitter, and cost-per-request across CoreWeave's compute stack. * Contribute to implementing and ...
Senior Software Engineer - Perf and Benchmarking
Bellevue, WA · On-site
$182K - $242K/yr
Develop and enhance Kubernetes-native benchmarking services that measure latency, throughput, jitter, and cost-per-request across CoreWeave's compute stack. * Contribute to implementing and ...
Sr. Software Engineer - Perf and Benchmarking
Bellevue, WA · On-site
$139K - $204K/yr
Implement and maintain benchmarking workflows for end-to-end MLPerf Training and Inference runs, including workload setup, cluster configuration, runbooks, and result validation. * Lead design ...
Sr. Software Engineer - Perf and Benchmarking
Bellevue, WA · On-site
$139K - $204K/yr
Implement and maintain benchmarking workflows for end-to-end MLPerf Training and Inference runs, including workload setup, cluster configuration, runbooks, and result validation. * Lead design ...
Benchmarking Program Coordinator
Irvine, CA · On-site
$36 - $39/hr
Drive product excellence by equipping product development and manufacturing engineering teams with data-driven insights derived from competitive benchmarking, market analysis, and comprehensive ...
Benchmarking Program Coordinator
Irvine, CA · On-site
$36 - $39/hr
Drive product excellence by equipping product development and manufacturing engineering teams with data-driven insights derived from competitive benchmarking, market analysis, and comprehensive ...
Sr. Software Engineer - Perf and Benchmarking
Bellevue, WA · On-site +1
$139K - $204K/yr
Implement and maintain benchmarking workflows for end-to-end MLPerf Training and Inference runs, including workload setup, cluster configuration, runbooks, and result validation. * Lead design ...
Sr. Software Engineer - Perf and Benchmarking
Bellevue, WA · On-site +1
$139K - $204K/yr
Implement and maintain benchmarking workflows for end-to-end MLPerf Training and Inference runs, including workload setup, cluster configuration, runbooks, and result validation. * Lead design ...
Job Benchmarking information
See salary details
$51K - $56.7K
6% of jobs
$56.7K - $62.5K
9% of jobs
$62.5K - $68.2K
9% of jobs
$69K is the 25th percentile. Wages below this are outliers.
$68.2K - $73.9K
12% of jobs
The median wage is $78.1K / yr.
$73.9K - $79.6K
20% of jobs
$85.2K is the 75th percentile. Wages above this are outliers.
$79.6K - $85.4K
20% of jobs
$85.4K - $91.1K
12% of jobs
$91.1K - $96.8K
6% of jobs
$96.8K - $102.5K
3% of jobs
$102.5K - $108.3K
1% of jobs
$108.3K - $114K
2% of jobs
$51K
$80.4K
$114K
How much do job benchmarking jobs pay per year?
What is the difference between Job Benchmarking vs Job Analysis?
| Aspect | Job Benchmarking | Job Analysis |
|---|---|---|
| Purpose | Compare jobs across organizations to establish salary standards and industry benchmarks | Identify specific duties, responsibilities, and requirements of a particular job |
| Focus | External comparison and market positioning | Internal understanding of a specific role |
| Data Used | Salary data, industry standards, market trends | Job duties, skills, work environment, qualifications |
| Application | Compensation planning, HR strategy, market competitiveness | Job design, recruitment, performance evaluation |
While Job Benchmarking compares roles across organizations to set competitive salary standards, Job Analysis focuses on understanding the specific duties and requirements of a single job within an organization. Both are essential HR tools but serve different purposes in workforce management.
Full-time
Posted 15 days ago
Job description
We are hiring Engineers focused on AI Model Evaluation to build the systems that ensure multimodal AI behaves reliably, consistently, and predictably as it moves from research into production. This position focuses on evaluating generative and vision-based models through automated benchmarking, dataset-driven testing, and performance validation pipelines.
You will work at the intersection of applied science, infrastructure, and product - helping define how we measure realism, consistency, and quality across image, video, and multimodal AI systems.
Why This Role Exists
Modern AI evaluation extends beyond pass/fail testing. Multimodal generative systems require:
- benchmarking across visual realism, pose consistency, and identity preservation,
- automated regression detection across model checkpoints,
- scalable evaluation pipelines integrated into continuous deployment workflows.
We are building evaluation systems where research velocity and product reliability must coexist. This role is for engineers interested in defining how quality is measured in generative AI systems.
What you'll do
- Build automated evaluation pipelines for multimodal AI models.
- Benchmark diffusion models, vision systems, and generative workflows.
- Validate model checkpoints and detect regressions across versions.
- Develop evaluation metrics for realism, consistency, and performance.
- Integrate evaluation tooling into CI/CD workflows.
- Collaborate with ML researchers and infrastructure teams to ensure production readiness.
- Analyze failure modes and propose evaluation strategies.
Core Areas & Tooling
Candidates should be familiar with or interested in:
- LLM, VLM, or Stable Diffusion model evals
- Image/Video benchmarking techniques
- Multimodal evaluation frameworks
- dataset-driven testing workflows
- research experiment validation pipelines
Qualifications
- Degree in Computer Science, AI, Engineering, or comparable combination of education and practical experience.
- Strong programming skills in Python.
- Familiarity with object-oriented programming (C++, Java, Python, or similar).
- Strong data structures and algorithms fundamentals.
- Understanding of machine learning experimentation workflows.
Preferred Qualifications
- Experience evaluating vision or generative models.
- Familiarity with HuggingFace ecosystem or open-source ML toolkits.
- Experience building automated test frameworks or benchmarking tools.
- Knowledge of diffusion models or multimodal architectures.
Experience with data analysis tools (NumPy, Pandas, visualization libraries).
SPREEAI is a fast-growing, innovative AI company at the forefront of fashion and e-commerce, revolutionizing how consumers engage with fashion through lifelike photorealistic try-on technology and hyper-personalized shopping experiences. Our mission is to redefine the retail landscape with cutting-edge AI solutions that blend high fashion and technology. We thrive in a dynamic, fast-paced environment where creativity meets technology to drive real impact. If you are passionate about innovation and shaping the future of fashion, SPREEAI offers a platform to make your mark.