ZipRecruiter

Log In

1

Machine Learning Infrastructure Jobs (NOW HIRING)

next page

Showing results 1-20

People also search for

All JobsMachine Learning Infrastructure Jobs

Machine Learning Infrastructure information

See salary details

$15

$28

$52

How much do machine learning infrastructure jobs pay per hour?

As of Jul 7, 2026, the average hourly pay for machine learning infrastructure in the United States is $28.01, according to ZipRecruiter salary data. Most workers in this role earn between $21.88 and $30.29 per hour, depending on experience, location, and employer.

What is the difference between Machine Learning Infrastructure vs Data Engineer?

AspectMachine Learning InfrastructureData Engineer
Required CredentialsBachelor's in CS, experience with ML toolsBachelor's in CS, experience with data pipelines
Work EnvironmentFocus on ML systems, cloud platformsData pipelines, database management
Employer & Industry UsageTech companies, AI startupsAny industry with data needs, tech firms
Search & Comparison IntentUnderstanding ML system setupBuilding data pipelines

Machine Learning Infrastructure specialists focus on deploying and maintaining systems that support machine learning models, often working with cloud platforms and ML tools. Data Engineers build and manage data pipelines and databases, supporting data collection and processing. While both roles require technical skills and overlap in data handling, Machine Learning Infrastructure is more centered on ML system deployment, whereas Data Engineers focus on data architecture and pipelines.

Will MLE be replaced by AI?

Machine Learning Engineers (MLEs) design, develop, and maintain AI systems, and their role involves understanding complex algorithms, data pipelines, and infrastructure. While AI automation tools can assist with certain tasks, MLEs are essential for building, optimizing, and overseeing AI models and infrastructure, making complete replacement unlikely in the near term.

How much do ML infra engineers make?

ML infrastructure engineers typically earn between $100,000 and $160,000 annually, depending on experience, location, and company size. Senior roles or those with specialized skills in cloud platforms and distributed systems can earn higher salaries, often exceeding $180,000. Compensation may also include bonuses and stock options in tech companies.

What are the typical challenges faced by professionals working in Machine Learning Infrastructure roles?

Professionals in Machine Learning Infrastructure often encounter challenges related to scaling systems to handle large datasets, ensuring model reproducibility, and maintaining efficient workflows for both development and deployment. Collaborating closely with data scientists, software engineers, and DevOps teams is crucial to address issues like version control, resource allocation, and performance optimization. Staying updated on evolving tools and cloud platforms is also essential, as the landscape changes rapidly and impacts system design and integration.

What engineer makes $500,000 a year?

Senior machine learning engineers and infrastructure engineers with extensive experience, advanced skills in distributed systems, and expertise in tools like TensorFlow or PyTorch can earn salaries around or above $500,000 annually, especially in high-cost-of-living areas or large tech companies. These roles often require advanced degrees, certifications, and leadership responsibilities.

What are the key skills and qualifications needed to thrive in Machine Learning Infrastructure, and why are they important?

To excel in Machine Learning Infrastructure, you need a solid background in computer science, software engineering, and distributed systems, often supported by experience in deploying and scaling machine learning models. Familiarity with cloud platforms (like AWS, GCP, or Azure), containerization tools (such as Docker and Kubernetes), and ML workflow systems (e.g., TensorFlow Extended, MLflow) is crucial. Strong problem-solving skills, collaboration, and the ability to communicate technical concepts effectively help you stand out in this field. These skills ensure scalable, reliable, and efficient deployment of ML solutions, enabling organizations to leverage machine learning at production scale.

What is machine learning infrastructure?

Machine learning infrastructure refers to the hardware, software, and tools needed to develop, train, deploy, and maintain machine learning models. It includes computing resources like GPUs and cloud platforms, as well as data storage, version control, and automation systems to support efficient model development and scaling.
More about Machine Learning Infrastructure jobs
What job categories do people searching Machine Learning Infrastructure jobs look for? The top searched job categories for Machine Learning Infrastructure jobs are:
Machine Learning Infrastructure jobs near you
Infographic showing various Machine Learning Infrastructure job openings in the United States as of July 2026, with employment types broken down into 95% Full Time, 3% Part Time, and 2% Contract. Highlights an 86% Physical, 4% Hybrid, and 10% Remote job distribution, with an average salary of $58,269 per year, or $28 per hour.
Machine Learning Infrastructure Engineer

Machine Learning Infrastructure Engineer

Astera Labs

San Jose, CA • On-site

$140K - $165K/yr

Other

Posted 20 days ago

Job description

Machine Learning Infrastructure Engineer

Location: San Jose, CA
Experience: 1-5 years
Team: Applied AI

The role

We're hiring a Machine Learning Infrastructure Engineer to build the runtime, platform, and operational backbone for modern AI systems. This role is for someone who wants to work on the systems behind the systems: model access layers, routing, serving paths, telemetry, observability, evaluation infrastructure, and the controls needed to make fast-moving AI work reliable in practice.

This is a platform role, but not in the old sense. The work is tightly coupled to how modern AI systems are actually built and used: multiple model providers, agent runtimes, skill and tool layers, inference telemetry, cost-aware routing, AI spend visibility, and governance that is strong enough for real internal adoption.

What you'll do
  • Build and improve internal AI infrastructure for LLM applications, agents, retrieval systems, and model-backed engineering workflows.
  • Own inference deployment paths across managed and self-serve environments, including access control, monitoring, and operational reliability.
  • Build platform layers such as model gateways, routing, runtime integrations, telemetry, and controls for safe execution at scale.
  • Develop AI Ops capabilities across evaluation, release readiness, observability, incident triage, regression detection, and cost monitoring.
  • Build dashboards, tracing, logging, and alerting for production AI systems, including spend and usage visibility across tools and teams.
  • Improve performance and unit economics through routing, caching, batching, failover, and latency/cost optimization.
  • Create reusable APIs, SDKs, and platform abstractions that make AI systems easier to deploy, evaluate, govern, and operate.
What we're looking for
  • 1-5 years of experience in software engineering, ML infrastructure, MLOps, platform engineering, or related backend/infrastructure roles.
  • Strong Python plus strong systems instincts.
  • Experience with AWS or GCP and real production service ownership.
  • Familiarity with inference deployments, model APIs, gateways, serving systems, or runtime infrastructure for LLM/ML workloads.
  • Experience with observability, telemetry, reliability engineering, and incident response.
  • Understanding of eval systems, release workflows, retrieval-backed systems, and debugging non-deterministic AI behavior.
  • Ability to translate messy platform needs into scalable internal infrastructure.
What strong candidates often look like

They have built or operated systems where latency, routing, cost, telemetry, and reliability actually matter. They understand that modern AI infrastructure is not just about getting a model endpoint running. It is about building the runtime, visibility, controls, and developer experience that let an applied AI team move fast without losing quality or trust.

Why this role is interesting

The team is building AI-ready infrastructure in the most literal sense: observability, access control, AI spend tracking, secure managed platforms, skill/tool infrastructure, and telemetry that spans requests, tools, models, and outcomes. If you want to work on the platform layer that makes modern agentic systems possible - and do it in a setting where the downstream users are serious engineers with high expectations - this is that role.

The base pay compensation range for this role is between $140,000 - $165,000

Apply