1

Inference Engineer Jobs (NOW HIRING)

Senior Inference Engineer

Palo Alto, CA · On-site

$200K - $300K/yr

Senior Inference Engineer AI Video Generation Company (Stealth) | Palo Alto, CA | Hybrid About the Role We are seeking a Senior Inference Engineer to accelerate the performance of our AI-driven video ...

Inference Engineer

San Francisco, CA · On-site

$180K - $250K/yr

About the Role We're hiring an Inference Engineer to advance our mission of building real-time multimodal intelligence. Your Impact * Design and build low latency, scalable, and reliable model ...

AI Inference Engineer

Burlingame, CA · On-site

$110K - $270K/yr

The AI Inference Engineer at Quadric will [1] port AI models to Quadric platform; [2] optimize the model deployment for efficient inference; [3] profile and benchmark the model performance. This ...

AI Inference Engineer

Burlingame, CA · On-site

$110K - $270K/yr

The AI Inference Engineer at Quadric will [1] port AI models to Quadric platform; [2] optimize the model deployment for efficient inference; [3] profile and benchmark the model performance. This ...

AI Inference Engineer

Burlingame, CA · On-site

$110K - $270K/yr

The AI Inference Engineer at Quadric will [1] port AI models to Quadric platform; [2] optimize the model deployment for efficient inference; [3] profile and benchmark the model performance. This ...

The Audio Inference Engineer will work on optimizing audio inference serving efficiency and advancing core audio model serving metrics while collaborating with training and serving infrastructure ...

The Audio Inference Engineer will work on optimizing audio inference serving efficiency and advancing core audio model serving metrics while collaborating with training and serving infrastructure ...

About the Role We're looking for a GPU Inference Engineer to contribute to improvements in model serving efficiency for our Robotics research. This is a high-impact role where you'll drive ...

We are specifically seeking an expert in high-performance LLM serving systems and inference optimization. In this role, you will push the boundaries of how large language models are served. What You ...

New

We are specifically seeking an expert in high-performance LLM serving systems and inference optimization. In this role, you will push the boundaries of how large language models are served. What You ...

New

Applied AI Inference Engineer

New York, NY · On-site +1

$165K - $330K/yr

THE ROLE As an Applied AI Inference Engineer at Baseten, you will partner directly with customers to architect, build, and deploy high-scale production AI applications on Baseten's platform. You'll ...

next page

Showing results 1-20

Inference Engineer information

See salary details

$44K

$106.4K

$173.5K

How much do inference engineer jobs pay per year?

As of Jul 8, 2026, the average yearly pay for inference engineer in the United States is $106,386.00, according to ZipRecruiter salary data. Most workers in this role earn between $76,000.00 and $132,500.00 per year, depending on experience, location, and employer.
Infographic showing various Inference Engineer job openings in the United States as of July 2026, with employment types broken down into 95% Full Time, 2% Part Time, and 3% Contract. Highlights an 87% Physical, 4% Hybrid, and 9% Remote job distribution, with an average salary of $106,386 per year, or $51.1 per hour.

Senior Inference Engineer

DEEPREC.AI

Palo Alto, CA • On-site

$200K - $300K/yr

Full-time

Posted 4 days ago


Job description

Senior Inference Engineer AI Video Generation Company (Stealth) | Palo Alto, CA | Hybrid
About the Role We are seeking a Senior Inference Engineer to accelerate the performance of our AI-driven video generation products. In this highly technical role, you will operate at the intersection of cutting-edge inference acceleration, GPU parallelism, advanced model deployment, and video generation technologies. Your expertise will drive significant improvements to model speed and efficiency, ensuring our creative AI systems deliver industry-leading user experiences at scale.
You will design and optimize inference pipelines, implement state-of-the-art acceleration techniques, and work closely with researchers and engineers across the team to push the boundaries of what's possible in real-time AI deployment. Your efforts will play a foundational role in powering the next generation of our video and language models.
What You'll Do
  • Accelerate Inference: Lead and implement advanced inference acceleration techniques, including attention optimization and quantization for efficient model serving.
  • Maximize GPU Parallelism: Engineer and optimize GPU strategies across tensor, sequence, and pipeline parallelism (TP, SP, PP) for maximal efficiency and scalability.
  • Programming for Performance: Develop and optimize high-performance computing kernels and distributed workloads using CUDA and NCCL.
  • Advance AI Deployment: Collaborate with research and engineering teams to bring state-of-the-art video generation and large language models into production.
  • Improve Training Efficiency: Contribute to improvements in model training speed, stability, and resource utilization as part of our deployment lifecycle. (Bonus)
  • Technical Excellence: Drive rigorous code reviews, participate in technical discussions, and mentor fellow engineers on best practices in inference and GPU programming.
What We're Looking For
  • Experience: 5 years of engineering experience, with a strong track record in inference acceleration and model deployment at scale.
  • Inference Mastery: Proven expertise in inference optimization, including quantization, attention acceleration, and deep learning compiler stacks.
  • GPU and Parallelism: Deep knowledge of GPU programming (CUDA, NCCL) and experience with SP, TP, PP, and other forms of parallelism for distributed inference.
  • AI Domain Knowledge: Familiarity with video generation models and large language models (LLMs).
  • Collaboration: Strong cross-discipline communication skills; able to drive shared goals across research and engineering functions.
  • Ownership Mindset: Self-driven, solutions-oriented, and capable of managing ambiguity in a fast-paced startup environment.
Nice to Have
  • Experience with high-throughput video or real-time streaming model deployment.
  • Familiarity with distributed training and optimization toolkits.
  • Contributions to open source projects in AI infrastructure or deep learning compilers.
  • Startup or rapid prototyping experience.
What We Offer
  • Competitive salary commensurate with AI industry benchmarks.
  • Equity in a fast-growing company shaping the future of generative AI.
  • Comprehensive health benefits, monthly stipends, and company retreats.
  • A collaborative, in-office culture focused on building and shipping together.
About the Company A well-funded, early-stage AI video generation startup headquartered in Palo Alto, CA. The team is building technology to make video creation seamless, intuitive, and universally accessible through the transformative power of AI. Tight-knit and highly energetic, the company values efficiency, intellectual curiosity, and the ambition to make a meaningful impact on the world.