ZipRecruiter

Log In

1

Machine Learning Infrastructure Engineer Jobs (NOW HIRING)

Talanto

$79K - $104K/yr

... Infrastructure Engineer, Model Inference at • • • • • • • , you'll play a pivotal role in building and optimizing the core inference infrastructure that powers our machine learning ...

Electronic Arts

Machine Learning Engineer III

Kirkland, WA · On-site

$122K - $158K/yr

Together, we power the platforms, AI-driven tools, live services, and infrastructure that ensure ... The Senior Machine Learning Engineer will report to the Senior Manager, EA Player Security Data ...

previous page

Showing results 41-60

All JobsTechnology JobsData JobsData Scientist JobsMachine Learning Infrastructure Engineer Jobs

Machine Learning Infrastructure Engineer information

See salary details

$46.5K

$127.1K

$182K

How much do machine learning infrastructure engineer jobs pay per year?

As of Sep 12, 2026, the average yearly pay for machine learning infrastructure engineer in the United States is $127,066.00, according to ZipRecruiter salary data. Most workers in this role earn between $107,500.00 and $141,000.00 per year, depending on experience, location, and employer.

What is a machine learning infrastructure engineer?

A Machine Learning Infrastructure Engineer designs, builds, and maintains the systems that support the development and deployment of machine learning models. This includes managing data pipelines, optimizing model training and inference, and ensuring scalability and reliability in production environments. They work closely with data scientists, ML engineers, and DevOps teams to create efficient workflows and infrastructure. Key technologies often include cloud platforms, containerization, orchestration tools, and distributed computing frameworks.

What are the key skills and qualifications needed to thrive as a machine learning infrastructure engineer?

To thrive as a Machine Learning Infrastructure Engineer, you need a strong background in computer science, cloud computing, distributed systems, and experience with machine learning frameworks, often supported by a degree in a related field. Familiarity with tools such as Docker, Kubernetes, Terraform, as well as cloud platforms like AWS, GCP, or Azure, and certifications in cloud or DevOps technologies are highly valued. Strong problem-solving abilities, effective communication, and collaboration skills help engineers work seamlessly with data scientists and cross-functional teams. These skills are essential to design, implement, and maintain robust, scalable infrastructure that enables efficient machine learning development and deployment.

What are some common challenges faced by machine learning infrastructure engineers, and how can these be addressed on the job?

Machine Learning Infrastructure Engineers often face challenges such as ensuring infrastructure scalability, managing resource allocation, and maintaining system reliability while supporting rapid experimentation by data science teams. Balancing the needs for flexibility in research environments with production-grade stability requires a deep understanding of both engineering best practices and the unique requirements of machine learning workflows. Collaboration with data scientists, clear communication about infrastructure capabilities, and staying current with fast-evolving technologies are key strategies for success. Most companies encourage ongoing learning and provide opportunities to contribute to architecture decisions, which makes this a rewarding environment for problem-solvers and innovators.

More about Machine Learning Infrastructure Engineer jobs

What cities are hiring for Machine Learning Infrastructure Engineer jobs?

Cities with the most Machine Learning Infrastructure Engineer job openings:

What states have the most Machine Learning Infrastructure Engineer jobs?

States with the most job openings for Machine Learning Infrastructure Engineer jobs include:

What are popular job titles related to Machine Learning Infrastructure Engineer jobs?

For Machine Learning Infrastructure Engineer jobs, the most frequently searched job titles are:

Machine Learning Infrastructure Engineer jobs near you
Infographic showing various Machine Learning Infrastructure Engineer job openings in the United States as of September 2026, with employment types broken down into 1% Internship, 1% As Needed, 74% Full Time, 22% Part Time, and 2% Contract. Highlights an 83% Physical, 2% Hybrid, and 15% Remote job distribution, with an average salary of $127,066 per year, or $61.1 per hour.

Machine Learning Infrastructure Engineer, Model Inference

On-site

Talanto
See more jobs at Talantoexternal

$79K - $104K/yr

Other

Medical, Dental, Vision, Retirement, PTO

Posted 12 days ago

Key responsibilities

  • Build, deploy, and maintain scalable Kubernetes clusters for AI model inference and training.

  • Develop, optimize, and maintain ML model serving infrastructure to ensure high performance and low latency.

  • Collaborate with ML and product teams to scale backend infrastructure for AI-driven products, focusing on model deployment and compute efficiency.

Job description

18,416 – 21,666 $

••••••• was founded in 2018 with the mission of powering deeper understanding in healthcare. Our AI-powered platform was purpose-built for medical conversations, improving clinical documentation efficiencies while enabling clinicians to focus on what matters most—their patients.

Our enterprise-grade technology transforms patient-clinician conversations into structured clinical notes in real-time, with deep EMR integrations. Powered by Linked Evidence and our purpose-built, auditable AI, we are the only company that maps AI-generated summaries to ground truth, helping providers quickly trust and verify the output. As pioneers in generative AI for healthcare, we are setting the industry standards for the responsible deployment of AI across health systems.

We are a growing team of practicing MDs, AI scientists, PhDs, creatives, technologists, and engineers working together to empower people and make care make more sense. We have offices located in the Mission District in San Francisco, the SoHo neighborhood of New York, and East Liberty in Pittsburgh.

The Role

As an ML Infrastructure Engineer, Model Inference at ••••••• , you’ll play a pivotal role in building and optimizing the core inference infrastructure that powers our machine learning models. Your work will be instrumental in enhancing the scalability, efficiency, and performance of our AI-driven solutions. You will work with our Infrastructure and Research teams to build, deploy, optimize and orchestrate across our AI models.

What You'll Do

Design, deploy and maintain scalable Kubernetes clusters for AI model inference and training

Develop, optimize, and maintain ML model serving infrastructure, ensuring high-performance and low-latency.

Collaborate with ML and product teams to scale backend infrastructure for AI-driven products, focusing on model deployment, throughput optimization, and compute efficiency.

Optimize compute-heavy workflows and enhance GPU utilization for ML workloads.

Build a robust model API orchestration system

Collaborate with leadership to define and implement strategies for scaling infrastructure as the company grows, ensuring long-term efficiency and performance.

What You’ll Bring

5+ years of experience in building and deploying machine learning models in production environments.

Deep understanding of container orchestration and distributed systems architecture

Expertise in Kubernetes administration, including custom resource definitions, operators, and cluster management

Experience developing APIs and managing distributed systems for both batch and real-time workloads

Excellent communication skills, with the ability to interface between research and product engineering

Ideally, You Have

Expertise with model serving frameworks such as NVIDIA Triton Server, VLLM, TRT-LLM and so on.

Expertise with ML toolchains such as PyTorch, Tensorflow or distributed training and inference libraries.

Familiarity with GPU cluster management and CUDA optimization

Knowledge of infrastructure as code (Terraform, Ansible) and GitOps practices

Experience with container registries, image optimization, and multi-stage builds for ML workloads

Experience orchestrating across ASR models or LLM models for building various GenAI applications

At ••••••• , we’re transforming healthcare delivery experiences with generative AI, enabling clinicians and patients to connect in deeper, more meaningful ways. Our mission is clear: to power deeper understanding in healthcare. We’re driving real, lasting change, with millions of medical conversations processed each month.

Joining ••••••• means stepping into a fast-paced, high-growth startup where your contributions truly make a difference. Our culture requires extreme ownership—every employee has the ability to (and is expected to) make an impact on our customers and our business.

Beyond individual impact, you will have the opportunity to work alongside a team of curious, high-achieving people in a supportive environment where success is shared, growth is constant, and feedback fuels progress. At ••••••• , it’s not just what we do—it’s how we do it. Every decision is rooted in empathy, always prioritizing the needs of clinicians and patients.

We’re committed to supporting your growth, both professionally and personally. Whether it's flexible work hours, an inclusive culture, or ongoing learning opportunities, we are here to help you thrive and do the best work of your life.

If you are ready to make a meaningful impact alongside passionate people who care deeply about what they do, ••••••• is the place for you.

How we take care of Abridgers:
  • Generous Time Off: 14 paid holidays, flexible PTO for salaried employees, and accrued time off for hourly employees
  • Comprehensive Health Plans: Medical, Dental, and Vision coverage for all full-time employees and their families.
  • Paid Parental Leave: Generous paid parental leave for all full-time employees.
  • Family Forming Benefits: Resources and financial support to help you build your family.
  • 401(k) Matching: Contribution matching to help invest in your future.
  • Personal Device Allowance: Tax free funds for personal device usage.
  • Pre-tax Benefits: Access to Flexible Spending Accounts (FSA) and Commuter Benefits.
  • Lifestyle Wallet: Monthly contributions for fitness, professional development, coworking, and more.
  • Mental Health Support: Dedicated access to therapy and coaching to help you reach your goals.
  • Sabbatical Leave: Paid Sabbatical Leave after 5 years of employment.
  • Compensation and Equity: Competitive compensation and equity grants for full time employees.
Equal Opportunity Employer

••••••• is an equal opportunity employer and considers all qualified applicants equally without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran status, or disability.

We're committed to providing reasonable accommodations throughout the interview process. Once you submit your application, we'll follow up with details on how to request an accommodation for interviewing, completing any assessments, or otherwise participating in the selection process.

#J-18808-Ljbffr

Apply