ZipRecruiter

Log In

1

Ml Infrastructure Jobs (NOW HIRING)

Clera

ML Infrastructure Engineer

San Mateo, CA · On-site

$122K - $160K/yr

About the Role This is a hands-on ML Infrastructure Engineer role at an early-stage enterprise AI startup, where you'll own the end-to-end inference and model-serving infrastructure that keeps ...

Lattice, Inc.

ML Infrastructure Engineer

San Francisco, CA · On-site

$126K - $166K/yr

Engineering San Francisco Full-time $200,000 - $280,000 About the Role Join our ML Infrastructure team to build the systems that train, deploy, and serve our AI models at scale. You'll work at the ...

Gridware

ML Infrastructure Engineer

San Francisco, CA

$190K - $250K/yr

Role Description As a Senior ML Infrastructure Engineer, you will work directly in the Automation org with the core ML, Ops, and Analytics teams to help improve and build out the infrastructure ...

Gridware

ML Infrastructure Engineer

San Francisco, CA

$190K - $250K/yr

Role Description As a Senior ML Infrastructure Engineer, you will work directly in the Automation org with the core ML, Ops, and Analytics teams to help improve and build out the infrastructure ...

SpaceXAI

ML Infrastructure Engineer

Palo Alto, CA · On-site

$180K - $440K/yr

As an ML Infrastructure Engineer, you will play a pivotal role in building and optimizing the reliable, high-performance ML platform that powers recommendations on X. We're looking for exceptional ...

Gridware

ML Infrastructure Engineer

San Francisco, CA · On-site

$190K - $250K/yr

Role Description As a Senior ML Infrastructure Engineer, you will work directly in the Automation org with the core ML, Ops, and Analytics teams to help improve and build out the infrastructure ...

SpaceXAI

ML Infrastructure Engineer

Palo Alto, CA · On-site

$180K - $440K/yr

As an ML Infrastructure Engineer, you will play a pivotal role in building and optimizing the reliable, high-performance ML platform that powers recommendations on X. We're looking for exceptional ...

Pantera Capital

ML Infrastructure Engineer

Palo Alto, CA · On-site

$180K - $440K/yr

As an ML Infrastructure Engineer, you will play a pivotal role in building and optimizing the reliable, high-performance ML platform that powers recommendations on X. We're looking for exceptional ...

SpaceXAI

ML Infrastructure Engineer

Palo Alto, CA · On-site

$180K - $440K/yr

As an ML Infrastructure Engineer, you will play a pivotal role in building and optimizing the reliable, high-performance ML platform that powers recommendations on X. We're looking for exceptional ...

Socket.dev

ML Infrastructure Engineer

Palo Alto, CA · On-site

$180K - $440K/yr

As an ML Infrastructure Engineer, you will play a pivotal role in building and optimizing the reliable, high-performance ML platform that powers recommendations on X. We're looking for exceptional ...

Graphon.AI

ML Infrastructure Engineer

San Francisco, CA · On-site

$180K - $250K/yr

Experience working with machine learning infrastructure , such as training pipelines, inference/serving systems, data pipelines, or model deployment. * Familiarity with modern ML stacks (e.g ...

Sunday

ML Infrastructure Engineer

Redwood City, CA · On-site

$131K - $172K/yr

They are seeking an ML Infrastructure Engineer to build and shape foundational systems that accelerate the deployment of robots in homes, encompassing responsibilities in training infrastructure ...

Graphon.AI

ML Infrastructure Engineer

San Francisco, CA · On-site

$180K - $250K/yr

Experience working with machine learning infrastructure , such as training pipelines, inference/serving systems, data pipelines, or model deployment. * Familiarity with modern ML stacks (e.g ...

SambaNova

Senior ML Infrastructure Engineer

Austin, TX

$107K - $146K/yr

About the team The ML Infrastructure team builds and operates the inference stack that serves SambaNova's models on RDU accelerators, from request scheduling and caching through the public APIs in ...

zaimler

ML Infrastructure Engineer

San Mateo, CA

$122K - $160K/yr

Set up and scale inference/Ray Serve for ML and LLM model serving, integrated with our data analysis and agent workflows * Scale agent GPU infrastructure for concurrency and efficiency across ...

zaimler

ML Infrastructure Engineer

San Mateo, CA · On-site

$122K - $160K/yr

Set up and scale inference/Ray Serve for ML and LLM model serving, integrated with our data analysis and agent workflows * Scale agent GPU infrastructure for concurrency and efficiency across ...

zaimler

ML Infrastructure Engineer

San Mateo, CA · On-site

$122K - $160K/yr

Set up and scale inference/Ray Serve for ML and LLM model serving, integrated with our data analysis and agent workflows * Scale agent GPU infrastructure for concurrency and efficiency across ...

Sunday

ML Infrastructure Engineer

Redwood City, CA · On-site

$131K - $172K/yr

Required : • Strong software engineering and systems fundamentals • Experience building distributed systems or large-scale data pipelines • Hands-on experience with ML training infrastructure ...

Rebar

Senior ML Infrastructure Engineer

New York, NY · On-site

$118K - $161K/yr

Senior ML Infrastructure Engineer Background Rebar is building the next-generation operating system for commercial HVAC, electrical, and plumbing suppliers and subcontractors. Over the past year, our ...

Jobtailor

Software Engineer, ML Infrastructure

California, MO · On-site

$154K - $183K/yr

Design and optimize infrastructure systems for machine learning workloads at scale and drive reliability and efficiency improvements across Snapchat's ML Infrastructure * Build and enhance feature ...

next page

Showing results 1-20

All JobsTechnology JobsMl Infrastructure Jobs

Ml Infrastructure information

See salary details

$46.5K

$127.1K

$182K

How much do ml infrastructure jobs pay per year?

As of Sep 12, 2026, the average yearly pay for ml infrastructure in the United States is $127,066.00, according to ZipRecruiter salary data. Most workers in this role earn between $107,500.00 and $141,000.00 per year, depending on experience, location, and employer.

What is ML infrastructure?

ML Infrastructure refers to the underlying systems, tools, and processes that enable the development, deployment, and scaling of machine learning models. This includes data storage and management, computing resources, model training and serving environments, monitoring, and automation tools. ML Infrastructure ensures that data scientists and engineers can efficiently build, test, and maintain machine learning applications in a reliable and reproducible manner. It is a crucial foundation for organizations looking to operationalize AI and machine learning solutions at scale.

What are some common challenges faced by professionals working in ML infrastructure roles?

Professionals in ML Infrastructure often encounter challenges related to scaling systems to handle large volumes of data, ensuring reliable deployment pipelines, and maintaining reproducibility across different environments. They must also collaborate closely with data scientists and engineers to streamline workflows and address issues like version control and model monitoring. Staying updated with rapidly evolving tools and best practices is essential, and balancing stability with innovation is a frequent aspect of the role.

What are the key skills and qualifications needed to thrive as an ML infrastructure engineer, and why are they important?

To thrive as an ML Infrastructure Engineer, you need a strong background in software engineering, cloud computing, and machine learning concepts, often supported by a degree in computer science or a related field. Proficiency with containerization tools (like Docker and Kubernetes), cloud platforms (such as AWS, GCP, or Azure), and CI/CD systems is critical. Excellent problem-solving, collaboration, and communication skills help you efficiently work with data scientists and DevOps teams. These skills and qualities are vital for building scalable, reliable ML systems that support rapid experimentation and deployment in production environments.

What is the difference between Ml Infrastructure vs Data Engineer?

AspectML InfrastructureData Engineer
Required CredentialsBachelor's in CS, Data Science, or related; knowledge of cloud platformsBachelor's in CS, Software Engineering, or related; experience with databases and ETL tools
Work EnvironmentFocus on deploying and maintaining ML systems, cloud environments, and infrastructure toolsDesigning, building, and managing data pipelines and storage solutions
Industry UsageUsed in AI/ML teams to support model deployment and scalabilityUsed across data-driven organizations for data management and analytics

ML Infrastructure specialists focus on deploying, scaling, and maintaining machine learning systems and infrastructure, while Data Engineers primarily build and manage data pipelines and storage solutions. Both roles require technical skills and often collaborate, but their core responsibilities differ in focus and tools used.

More about Ml Infrastructure jobs

What cities are hiring for Ml Infrastructure jobs?

Cities with the most Ml Infrastructure job openings:

What states have the most Ml Infrastructure jobs?

States with the most job openings for Ml Infrastructure jobs include:

What are popular job titles for Ml Infrastructure?

Popular job titles for Ml Infrastructure:

Ml Infrastructure jobs near you
Infographic showing various Ml Infrastructure job openings in the United States as of September 2026, with employment types broken down into 1% Internship, 88% Full Time, 9% Part Time, and 2% Contract. Highlights an 82% Physical, 5% Hybrid, and 13% Remote job distribution, with an average salary of $127,066 per year, or $61.1 per hour.

ML Infrastructure Engineer

San Mateo, CA • On-site

Clera
1 - 10 employees
Learn more about Cleraexternal

$122K - $160K/yr

Other

Posted 25 days ago

Job description

About the Role

This is a hands-on ML Infrastructure Engineer role at an early-stage enterprise AI startup, where you'll own the end-to-end inference and model-serving infrastructure that keeps production AI agents running reliably and at scale. You'll sit at the intersection of ML and platform engineering, directly shaping the systems that power real-world, high-stakes deployments in regulated industries like insurance, banking, and healthcare.

What You'll Do

  • Own inference and model-serving infrastructure end to end, from design through production deployment.

  • Build and scale systems that enable AI agents to run reliably and efficiently under increasing concurrency.

  • Collaborate closely with ML and infrastructure teams to ensure seamless integration and performance optimization.

  • Identify infrastructure bottlenecks and drive cross-functional solutions across engineering teams.

What We're Looking For

  • 5+ years of experience building and operating ML inference systems, model-serving platforms, or ML infrastructure in production.

  • Hands-on experience designing and scaling inference-serving infrastructure using frameworks such as TensorFlow Serving, TorchServe, Triton, KServe, or equivalent custom systems.

  • Strong track record optimizing production ML systems for latency, throughput, and reliability at scale.

  • Experience with containerization and orchestration (Docker, Kubernetes) for deploying and scaling ML workloads.

  • Experience building distributed systems that handle concurrent requests and manage resource allocation under load.

  • Proficiency with observability and debugging tooling for production systems (e.g., Prometheus, Grafana, ELK, distributed tracing).

  • Cloud platform experience on AWS, GCP, or Azure for deploying and managing ML systems.

  • Proficiency in at least one systems or backend language — Python, Go, Rust, C++, or Java.

  • Nice to have: experience with knowledge graphs, semantic search, or graph databases (e.g., Neo4j, Amazon Neptune); real-time or low-latency inference systems; agentic or multi-step reasoning pipelines; enterprise data infrastructure or integration platforms.

Location

On-site in San Mateo, CA. No visa sponsorship is available for this role.

#J-18808-Ljbffr

Apply