ZipRecruiter

Log In

1

Ml Infrastructure Jobs (NOW HIRING)

zaimler

ML Infrastructure Engineer

San Mateo, CA · On-site

$122K - $160K/yr

Set up and scale inference/Ray Serve for ML and LLM model serving, integrated with our data analysis and agent workflows * Scale agent GPU infrastructure for concurrency and efficiency across ...

Rebar

Senior ML Infrastructure Engineer

New York, NY · On-site

$118K - $161K/yr

Senior ML Infrastructure Engineer Background Rebar is building the next-generation operating system for commercial HVAC, electrical, and plumbing suppliers and subcontractors. Over the past year, our ...

Sunday Robotics

ML Infrastructure Engineer

Redwood City, CA · On-site

$131K - $172K/yr

Hands‑on experience with ML training infrastructure, ideally PyTorch * Comfort reasoning about performance, memory, I/O, and GPU utilization * Experience managing training workloads (SLURM ...

Jobtailor

Software Engineer, ML Infrastructure

California, MO · On-site

$154K - $183K/yr

Design and optimize infrastructure systems for machine learning workloads at scale and drive reliability and efficiency improvements across Snapchat's ML Infrastructure * Build and enhance feature ...

Sunday Inc

ML Infrastructure Engineer

Redwood City, CA · On-site

$131K - $172K/yr

Hands-on experience with ML training infrastructure, ideally PyTorch * Comfort reasoning about performance, memory, I/O, and GPU utilization * Experience managing training workloads (SLURM ...

Cursor

Software Engineer, ML Infrastructure

Manhattan, NY · On-site

$190K - $226K/yr

About the role The ML Infrastructure team builds large-scale compute, storage, and software infrastructure to support Cursor's work building the world's best agentic coding model. We're looking for ...

Cursor

Software Engineer, ML Infrastructure

San Francisco, CA · On-site

$203K - $241K/yr

About the role The ML Infrastructure team builds large-scale compute, storage, and software infrastructure to support Cursor's work building the world's best agentic coding model. We're looking for ...

AppLovin

ML Infrastructure Engineer

Palo Alto, CA · On-site

$126K - $165K/yr

They are seeking an ML Infrastructure Engineer to design, develop, and maintain large-scale distributed systems while collaborating with various engineering teams to enhance their infrastructure and ...

SambaNova

Senior ML Infrastructure Engineer

Austin, TX · On-site

$200K - $275K/yr

About the team The ML Infrastructure team builds and operates the inference stack that serves SambaNova's models on RDU accelerators, from request scheduling and caching through the public APIs in ...

Zensors

AI/ML Infrastructure Engineer

San Francisco, CA · On-site

$126K - $166K/yr

The AI Infrastructure team at Zensors builds the engine that powers our visual sensing platform. We ... As a Machine Learning Engineer in ML Runtime & Optimization , you will develop technologies to ...

uRun

Founding Engineer - ML Infrastructure

San Francisco, CA · On-site

$126K - $166K/yr

As our ML Infrastructure and Platform Engineer, you will own the architecture and scaling of our GPU compute platform from the ground up. This is a founding technical hire with end-to-end ownership ...

New

Maven Robotics

ML Infrastructure Engineer

San Francisco, CA · On-site

$126K - $166K/yr

Build intuitive internal tools and abstractions that make complex infrastructure easy for engineers to use. * Lead technical and commercial discussions with cloud and ML compute providers, including ...

previous pagenext page

Showing results 21-40

All JobsTechnology JobsMl Infrastructure Jobs

Ml Infrastructure information

See salary details

$46.5K

$127.1K

$182K

How much do ml infrastructure jobs pay per year?

As of Sep 13, 2026, the average yearly pay for ml infrastructure in the United States is $127,066.00, according to ZipRecruiter salary data. Most workers in this role earn between $107,500.00 and $141,000.00 per year, depending on experience, location, and employer.

What is ML infrastructure?

ML Infrastructure refers to the underlying systems, tools, and processes that enable the development, deployment, and scaling of machine learning models. This includes data storage and management, computing resources, model training and serving environments, monitoring, and automation tools. ML Infrastructure ensures that data scientists and engineers can efficiently build, test, and maintain machine learning applications in a reliable and reproducible manner. It is a crucial foundation for organizations looking to operationalize AI and machine learning solutions at scale.

What are some common challenges faced by professionals working in ML infrastructure roles?

Professionals in ML Infrastructure often encounter challenges related to scaling systems to handle large volumes of data, ensuring reliable deployment pipelines, and maintaining reproducibility across different environments. They must also collaborate closely with data scientists and engineers to streamline workflows and address issues like version control and model monitoring. Staying updated with rapidly evolving tools and best practices is essential, and balancing stability with innovation is a frequent aspect of the role.

What are the key skills and qualifications needed to thrive as an ML infrastructure engineer, and why are they important?

To thrive as an ML Infrastructure Engineer, you need a strong background in software engineering, cloud computing, and machine learning concepts, often supported by a degree in computer science or a related field. Proficiency with containerization tools (like Docker and Kubernetes), cloud platforms (such as AWS, GCP, or Azure), and CI/CD systems is critical. Excellent problem-solving, collaboration, and communication skills help you efficiently work with data scientists and DevOps teams. These skills and qualities are vital for building scalable, reliable ML systems that support rapid experimentation and deployment in production environments.

What is the difference between Ml Infrastructure vs Data Engineer?

AspectML InfrastructureData Engineer
Required CredentialsBachelor's in CS, Data Science, or related; knowledge of cloud platformsBachelor's in CS, Software Engineering, or related; experience with databases and ETL tools
Work EnvironmentFocus on deploying and maintaining ML systems, cloud environments, and infrastructure toolsDesigning, building, and managing data pipelines and storage solutions
Industry UsageUsed in AI/ML teams to support model deployment and scalabilityUsed across data-driven organizations for data management and analytics

ML Infrastructure specialists focus on deploying, scaling, and maintaining machine learning systems and infrastructure, while Data Engineers primarily build and manage data pipelines and storage solutions. Both roles require technical skills and often collaborate, but their core responsibilities differ in focus and tools used.

More about Ml Infrastructure jobs

What cities are hiring for Ml Infrastructure jobs?

Cities with the most Ml Infrastructure job openings:

What states have the most Ml Infrastructure jobs?

States with the most job openings for Ml Infrastructure jobs include:

What are popular job titles for Ml Infrastructure?

Popular job titles for Ml Infrastructure:

Ml Infrastructure jobs near you
Infographic showing various Ml Infrastructure job openings in the United States as of September 2026, with employment types broken down into 1% Internship, 89% Full Time, 8% Part Time, and 2% Contract. Highlights an 81% Physical, 5% Hybrid, and 14% Remote job distribution, with an average salary of $127,066 per year, or $61.1 per hour.

ML Infrastructure Engineer

San Mateo, CA • On-site

zaimler
See more jobs at zaimlerexternal

$122K - $160K/yr

Full-time

Medical, Dental, Vision, Retirement

Re-posted 28 days ago

Job description

About zaimler
AI agents can't reason over data they don't understand. Enterprise data today is fragmented across dozens of systems with no shared context, meaning, or structure, and that's why most enterprise AI is failing. The shift from copilots to autonomous agents is creating an entirely new infrastructure layer, and we're building it.
zaimler is the context infrastructure for the agentic era: a platform that automatically discovers domain knowledge, maps relationships, and gives AI agents the semantic understanding to operate with precision at scale. Imagine knowledge graphs that support real-time inference, built for systems that need to reason, not just retrieve.
zaimler was founded by Biswajit Das (ex-VP Engineering, Truera), a Data Infra veteran and former Chief Architect at Visa, and Sofus Macskassy (ex-Director of Engineering, LinkedIn), who built one of the largest knowledge graphs in production in the industry at LinkedIn. We're growing and deploying with major enterprises across insurance, travel, and technology. If you want to build infrastructure that the next decade of enterprise AI runs on, we'd love to talk.
About the Role
You'll own our inference and model-serving infrastructure end to end. This isn't a research role. It's a build role: you're setting up and scaling the systems that let our agents actually run in production, fast and reliably, at increasing concurrency.
You report to Sofus and work closely with our ML and infra teams.
What You'll Own
  • Set up and scale inference/Ray Serve for ML and LLM model serving, integrated with our data analysis and agent workflows
  • Scale agent GPU infrastructure for concurrency and efficiency across multiple agent workloads
  • Optimize and improve the engine builder and model server that power scalable agent orchestration

What You Need
  • Proven ability to build scalable ML/AI platforms from scratch, end-to-end, for production use cases. You've owned a zero-to-one build before, or can show you're capable of it
  • Deep understanding of the inference stack: vLLM, KV cache, and the optimization layers underneath model serving
  • Experience building distributed systems for AI/ML workloads at scale, connecting them to real product or vertical integrations
  • 3+ years of relevant experience. We care about capability, not tenure

Nice to Have
  • Ray / Ray Serve experience
  • Familiarity with AIBrix

Why Join
  • A rare chance to shape both company and product direction as an early team engineer
  • Work alongside engineers and researchers from LinkedIn, Visa, Meta, and Branch
  • Onsite culture in San Mateo, built for deep collaboration and high-velocity building
  • Full benefits (medical, dental, vision, 401k)
  • We sponsor H-1B visas and assist with immigration

We value builders over résumés. If this role excites you but you don't check every box, we still want to hear from you. zaimler is an equal opportunity employer.
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

Apply