ZipRecruiter

Log In

1

Ml Infrastructure Jobs (NOW HIRING)

Gridware

ML Infrastructure Engineer

San Francisco, CA

$190K - $250K/yr

Role Description As a Senior ML Infrastructure Engineer, you will work directly in the Automation org with the core ML, Ops, and Analytics teams to help improve and build out the infrastructure ...

Gridware

ML Infrastructure Engineer

San Francisco, CA · On-site

$190K - $250K/yr

Role Description As a Senior ML Infrastructure Engineer, you will work directly in the Automation org with the core ML, Ops, and Analytics teams to help improve and build out the infrastructure ...

Gridware

ML Infrastructure Engineer

San Francisco, CA · On-site

$190K - $250K/yr

Role Description As a Senior ML Infrastructure Engineer, you will work directly in the Automation org with the core ML, Ops, and Analytics teams to help improve and build out the infrastructure ...

Graphon.AI

ML Infrastructure Engineer

San Francisco, CA · On-site

$180K - $250K/yr

Experience working with machine learning infrastructure , such as training pipelines, inference/serving systems, data pipelines, or model deployment. * Familiarity with modern ML stacks (e.g ...

Rebar

Senior ML Infrastructure Engineer

New York, NY · On-site

$118K - $161K/yr

Senior ML Infrastructure Engineer Background Rebar is building the next-generation operating system for commercial HVAC, electrical, and plumbing suppliers and subcontractors. Over the past year, our ...

Sunday Inc

ML Infrastructure Engineer

Redwood City, CA · On-site

$131K - $172K/yr

Hands-on experience with ML training infrastructure, ideally PyTorch * Comfort reasoning about performance, memory, I/O, and GPU utilization * Experience managing training workloads (SLURM ...

AppLovin

ML Infrastructure Engineer

Palo Alto, CA · On-site

$126K - $165K/yr

They are seeking an ML Infrastructure Engineer to design, develop, and maintain large-scale distributed systems while collaborating with various engineering teams to enhance their infrastructure and ...

AppLovin

ML Infrastructure Engineer

Palo Alto, CA · On-site

$126K - $165K/yr

The ML Infrastructure Engineer will design, develop, and maintain large-scale distributed systems while collaborating with various engineering teams to enhance the company's technology stack.

Maven Robotics

ML Infrastructure Engineer

San Francisco, CA

$126K - $166K/yr

Build intuitive internal tools and abstractions that make complex infrastructure easy for engineers to use. * Lead technical and commercial discussions with cloud and ML compute providers, including ...

next page

Showing results 1-20

All JobsMl Infrastructure Jobs

Ml Infrastructure information

See salary details

$46.5K

$127.1K

$182K

How much do ml infrastructure jobs pay per year?

As of Jul 14, 2026, the average yearly pay for ml infrastructure in the United States is $127,066.00, according to ZipRecruiter salary data. Most workers in this role earn between $107,500.00 and $141,000.00 per year, depending on experience, location, and employer.

What are some common challenges faced by professionals working in ML Infrastructure roles?

Professionals in ML Infrastructure often encounter challenges related to scaling systems to handle large volumes of data, ensuring reliable deployment pipelines, and maintaining reproducibility across different environments. They must also collaborate closely with data scientists and engineers to streamline workflows and address issues like version control and model monitoring. Staying updated with rapidly evolving tools and best practices is essential, and balancing stability with innovation is a frequent aspect of the role.

What is the difference between Ml Infrastructure vs Data Engineer?

AspectML InfrastructureData Engineer
Required CredentialsBachelor's in CS, Data Science, or related; knowledge of cloud platformsBachelor's in CS, Software Engineering, or related; experience with databases and ETL tools
Work EnvironmentFocus on deploying and maintaining ML systems, cloud environments, and infrastructure toolsDesigning, building, and managing data pipelines and storage solutions
Industry UsageUsed in AI/ML teams to support model deployment and scalabilityUsed across data-driven organizations for data management and analytics

ML Infrastructure specialists focus on deploying, scaling, and maintaining machine learning systems and infrastructure, while Data Engineers primarily build and manage data pipelines and storage solutions. Both roles require technical skills and often collaborate, but their core responsibilities differ in focus and tools used.

What are the key skills and qualifications needed to thrive as an ML Infrastructure Engineer, and why are they important?

To thrive as an ML Infrastructure Engineer, you need a strong background in software engineering, cloud computing, and machine learning concepts, often supported by a degree in computer science or a related field. Proficiency with containerization tools (like Docker and Kubernetes), cloud platforms (such as AWS, GCP, or Azure), and CI/CD systems is critical. Excellent problem-solving, collaboration, and communication skills help you efficiently work with data scientists and DevOps teams. These skills and qualities are vital for building scalable, reliable ML systems that support rapid experimentation and deployment in production environments.

What is ML Infrastructure?

ML Infrastructure refers to the underlying systems, tools, and processes that enable the development, deployment, and scaling of machine learning models. This includes data storage and management, computing resources, model training and serving environments, monitoring, and automation tools. ML Infrastructure ensures that data scientists and engineers can efficiently build, test, and maintain machine learning applications in a reliable and reproducible manner. It is a crucial foundation for organizations looking to operationalize AI and machine learning solutions at scale.
More about Ml Infrastructure jobs
What cities are hiring for Ml Infrastructure jobs? Cities with the most Ml Infrastructure job openings:
What states have the most Ml Infrastructure jobs? States with the most job openings for Ml Infrastructure jobs include:
What job categories do people searching Ml Infrastructure jobs look for? The top searched job categories for Ml Infrastructure jobs are:
Ml Infrastructure jobs near you
Infographic showing various Ml Infrastructure job openings in the United States as of July 2026, with employment types broken down into 95% Full Time, 3% Part Time, and 2% Contract. Highlights an 86% Physical, 4% Hybrid, and 10% Remote job distribution, with an average salary of $127,066 per year, or $61.1 per hour.
ML Infrastructure Engineer

ML Infrastructure Engineer

Mach9 Robotics Inc

San Francisco, CA • On-site

$160K - $200K/yr

Full-time

Re-posted 19 days ago

Job description

The role
At Mach9, ML infrastructure engineers build and maintain the systems that power production AI models for civil engineering and surveying. Our ML pipeline spans 10,000+ miles of labeled survey data, image segmentation networks, and 3D prediction models serving real-time inference to surveyors and engineers in the field.
This role is ideal for mid-career ML infrastructure engineers with experience building for both training and inference.
You'll build training pipelines that handle deep transformer models on hundreds of terabytes of 3D point cloud and image data. You'll also architect our inference infrastructure, delivering both heavy offline detection algorithms and real-time responsive inference that integrates directly with our CAD software.
Responsibilities
  • Design and build a centralized system for versioning training data, generated datasets, and model artifacts, with full lineage tracking from raw source data through to trained model outputs.
  • Develop and maintain reliable, reproducible ML training and data generation pipelines.
  • Refactor and harden existing training and data generation scripts into composable, testable, and maintainable components.
  • Create CI/CD workflows for validating data pipelines and model training runs, including automated correctness checks and regression detection.
  • Build tooling that enables ML engineers to launch, monitor, and debug training jobs with minimal friction.
  • Optimize and scale real-time model inference services to meet latency and throughput requirements in production, including profiling, batching strategies, and resource-efficient serving.
  • Own the deployment path from trained model artifact to production endpoint, ensuring reliable rollouts, rollback, and monitoring.

Requirements
  • 3+ years of work experience in relevant fields.
  • Bachelor's or Master's degree in Computer Science, Engineering, or equivalent experience.
  • Strong communication skills and the ability to work closely with ML researchers and engineers to understand their workflows and translate them into robust systems.
  • Experience designing and building data versioning, artifact management, or dataset lineage systems (e.g., DVC, LakeFS, Weights & Biases, or custom solutions).
  • Hands-on experience with ML pipeline orchestration tools (e.g., Airflow, Prefect, Metaflow, or similar).
  • Experience with model serving and inference optimization - profiling latency, reducing memory footprint, or scaling serving infrastructure to meet real-time constraints.
  • Ability to read and refactor ML training code - you don't need to design model architectures, but you need to understand what training pipelines are doing well enough to make them reliable.
  • Proficient with Python, PyTorch.

Bonus qualifications
  • Familiarity with AWS infrastructure services.
  • Experience with containerized ML workflows and GPU-accelerated training environments.
  • Experience with model optimization techniques (e.g., quantization, TensorRT, ONNX Runtime, distillation).
  • Knowledge of infrastructure-as-code tools (e.g., AWS CDK, Terraform).
  • Experience building or operating ML systems that handle large unstructured datasets (imagery, 3D data, sensor data).

Apply