ZipRecruiter

Log In

1

Ml Infrastructure Jobs (NOW HIRING)

SpaceXAI

ML Infrastructure Engineer

Palo Alto, CA · On-site

$180K - $440K/yr

As an ML Infrastructure Engineer, you will play a pivotal role in building and optimizing the reliable, high-performance ML platform that powers recommendations on X. We're looking for exceptional ...

Gridware

ML Infrastructure Engineer

San Francisco, CA

$190K - $250K/yr

Role Description As a Senior ML Infrastructure Engineer, you will work directly in the Automation org with the core ML, Ops, and Analytics teams to help improve and build out the infrastructure ...

Gridware

ML Infrastructure Engineer

San Francisco, CA · On-site

$190K - $250K/yr

Role Description As a Senior ML Infrastructure Engineer, you will work directly in the Automation org with the core ML, Ops, and Analytics teams to help improve and build out the infrastructure ...

Gridware

ML Infrastructure Engineer

San Francisco, CA · On-site

$190K - $250K/yr

Role Description As a Senior ML Infrastructure Engineer, you will work directly in the Automation org with the core ML, Ops, and Analytics teams to help improve and build out the infrastructure ...

SpaceXAI

ML Infrastructure Engineer

Palo Alto, CA · On-site

$180K - $440K/yr

As an ML Infrastructure Engineer, you will play a pivotal role in building and optimizing the reliable, high-performance ML platform that powers recommendations on X. We're looking for exceptional ...

SpaceXAI

ML Infrastructure Engineer

Palo Alto, CA

$180K - $440K/yr

As an ML Infrastructure Engineer, you will play a pivotal role in building and optimizing the reliable, high-performance ML platform that powers recommendations on X. We're looking for exceptional ...

Graphon.AI

ML Infrastructure Engineer

San Francisco, CA · On-site

$180K - $250K/yr

Experience working with machine learning infrastructure , such as training pipelines, inference/serving systems, data pipelines, or model deployment. * Familiarity with modern ML stacks (e.g ...

Graphon.AI

ML Infrastructure Engineer

San Francisco, CA · On-site

$180K - $250K/yr

Experience working with machine learning infrastructure , such as training pipelines, inference/serving systems, data pipelines, or model deployment. * Familiarity with modern ML stacks (e.g ...

Rebar

Senior ML Infrastructure Engineer

New York, NY · On-site

$118K - $161K/yr

Senior ML Infrastructure Engineer Background Rebar is building the next-generation operating system for commercial HVAC, electrical, and plumbing suppliers and subcontractors. Over the past year, our ...

Nebius

$91K - $119K/yr

... AI/ML infrastructure. Built by engineers, for engineers. From large-scale GPU orchestration to inference optimization, we own the hard problems across compute, storage, networking and applied AI.

Sunday Inc

ML Infrastructure Engineer

Redwood City, CA · On-site

$131K - $172K/yr

Hands-on experience with ML training infrastructure, ideally PyTorch * Comfort reasoning about performance, memory, I/O, and GPU utilization * Experience managing training workloads (SLURM ...

AppLovin

ML Infrastructure Engineer

Palo Alto, CA · On-site

$126K - $165K/yr

They are seeking an ML Infrastructure Engineer to design, develop, and maintain large-scale distributed systems while collaborating with various engineering teams to enhance their infrastructure and ...

AppLovin

ML Infrastructure Engineer

Palo Alto, CA · On-site

$126K - $165K/yr

The ML Infrastructure Engineer will design, develop, and maintain large-scale distributed systems while collaborating with various engineering teams to enhance the company's technology stack.

Maven Robotics

ML Infrastructure Engineer

San Francisco, CA

$126K - $166K/yr

Build intuitive internal tools and abstractions that make complex infrastructure easy for engineers to use. * Lead technical and commercial discussions with cloud and ML compute providers, including ...

next page

Showing results 1-20

All JobsMl Infrastructure Jobs

Ml Infrastructure information

See salary details

$46.5K

$127.1K

$182K

How much do ml infrastructure jobs pay per year?

As of Aug 3, 2026, the average yearly pay for ml infrastructure in the United States is $127,066.00, according to ZipRecruiter salary data. Most workers in this role earn between $107,500.00 and $141,000.00 per year, depending on experience, location, and employer.

What are some common challenges faced by professionals working in ML Infrastructure roles?

Professionals in ML Infrastructure often encounter challenges related to scaling systems to handle large volumes of data, ensuring reliable deployment pipelines, and maintaining reproducibility across different environments. They must also collaborate closely with data scientists and engineers to streamline workflows and address issues like version control and model monitoring. Staying updated with rapidly evolving tools and best practices is essential, and balancing stability with innovation is a frequent aspect of the role.

What is the difference between Ml Infrastructure vs Data Engineer?

AspectML InfrastructureData Engineer
Required CredentialsBachelor's in CS, Data Science, or related; knowledge of cloud platformsBachelor's in CS, Software Engineering, or related; experience with databases and ETL tools
Work EnvironmentFocus on deploying and maintaining ML systems, cloud environments, and infrastructure toolsDesigning, building, and managing data pipelines and storage solutions
Industry UsageUsed in AI/ML teams to support model deployment and scalabilityUsed across data-driven organizations for data management and analytics

ML Infrastructure specialists focus on deploying, scaling, and maintaining machine learning systems and infrastructure, while Data Engineers primarily build and manage data pipelines and storage solutions. Both roles require technical skills and often collaborate, but their core responsibilities differ in focus and tools used.

What are the key skills and qualifications needed to thrive as an ML Infrastructure Engineer, and why are they important?

To thrive as an ML Infrastructure Engineer, you need a strong background in software engineering, cloud computing, and machine learning concepts, often supported by a degree in computer science or a related field. Proficiency with containerization tools (like Docker and Kubernetes), cloud platforms (such as AWS, GCP, or Azure), and CI/CD systems is critical. Excellent problem-solving, collaboration, and communication skills help you efficiently work with data scientists and DevOps teams. These skills and qualities are vital for building scalable, reliable ML systems that support rapid experimentation and deployment in production environments.

What is ML Infrastructure?

ML Infrastructure refers to the underlying systems, tools, and processes that enable the development, deployment, and scaling of machine learning models. This includes data storage and management, computing resources, model training and serving environments, monitoring, and automation tools. ML Infrastructure ensures that data scientists and engineers can efficiently build, test, and maintain machine learning applications in a reliable and reproducible manner. It is a crucial foundation for organizations looking to operationalize AI and machine learning solutions at scale.
More about Ml Infrastructure jobs
What cities are hiring for Ml Infrastructure jobs? Cities with the most Ml Infrastructure job openings:
What states have the most Ml Infrastructure jobs? States with the most job openings for Ml Infrastructure jobs include:
What job categories do people searching Ml Infrastructure jobs look for? The top searched job categories for Ml Infrastructure jobs are:
Ml Infrastructure jobs near you
Infographic showing various Ml Infrastructure job openings in the United States as of July 2026, with employment types broken down into 67% Full Time, and 33% Contract. Highlights an 93% In-person, and 7% Remote job distribution, with an average salary of $127,066 per year, or $61.1 per hour.

ML Infrastructure Engineer

Bright Vision Technologies

Scottsdale, AZ • On-site, Remote

$100K - $150K/yr

Full-time

Posted 8 days ago

Job description

ML Infrastructure Engineer - Remote 
 
Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States. 
This is a fantastic opportunity to join an established and well-respected organization offering tremendous career growth potential. 
 
Job Title: ML Infrastructure Engineer
Location: 100% Remote (U.S.) 
Position Type: Full-time, Direct W2 
Salary Range: $100,000–$150,000 Annually 
Experience Required: 6+ years 
 
Sponsorship: U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates are encouraged to apply. We are unable to sponsor new H-1B visa petitions for this position. 
 
Job Summary 
We are seeking an AI Infrastructure Engineer to design, build, and operate the platform layer that powers large-scale AI training and inference workloads. The role focuses on GPU clusters, distributed training frameworks, scheduling, storage performance, and developer experience for ML engineers and researchers, with strong emphasis on reliability, efficiency, and cost control. The ideal candidate has built or operated production AI infrastructure at scale, understands the interaction between hardware, kernel, scheduler, and ML framework, and brings strong software engineering discipline to platform work. 
Key Responsibilities 
  • Design and operate GPU and accelerator infrastructure for training and inference, spanning on-prem clusters, cloud-managed services, and hybrid configurations. 
  • Build scheduling, queueing, and resource-sharing systems that maximize accelerator utilization across many teams. 
  • Integrate frameworks such as PyTorch, JAX, DeepSpeed, FSDP, Megatron-LM, and Ray Train into a unified platform offering. 
  • Operate high-performance storage systems and data pipelines that keep accelerators fed with training data at near-line-rate. 
  • Design networking architectures supporting RDMA, InfiniBand, NCCL, and high-bandwidth collective communication. 
  • Build observability for AI workloads including utilization, throughput, training stability, and failure-mode analytics. 
  • Implement checkpointing, restart, and fault-tolerance patterns for long-running training jobs at scale. 
  • Drive cost optimization across compute, storage, and networking through scheduling, spot capacity, and right-sizing. 
  • Develop developer tooling and paved-road workflows that let researchers launch experiments safely and efficiently. 
  • Partner with research and applied ML teams to plan capacity for upcoming training runs. 
  • Implement security controls, isolation, and access management for multi-tenant AI infrastructure. 
  • Drive automation across cluster provisioning, lifecycle management, and configuration enforcement. 
  • Maintain runbooks, capacity dashboards, and operational documentation for the AI platform. 
  • Stay current with AI infrastructure research, accelerator hardware, and emerging open-source AI tooling. 
Required Qualifications 
  • Bachelor’s or Master’s degree in Computer Science or a related field. 
  • Six or more years of experience in infrastructure, platform, or HPC engineering. 
  • Hands-on experience operating GPU clusters or large-scale ML training infrastructure. 
  • Strong proficiency in Python and at least one systems language such as Go or C++. 
  • Deep understanding of distributed training, accelerator architectures, and collective communication. 
  • Experience with Kubernetes, Slurm, Ray, or similar scheduling systems for ML workloads. 
  • Strong understanding of Linux internals, networking, and high-performance storage. 
  • Experience with at least one major cloud provider’s ML infrastructure offerings. 
  • Strong software engineering practices including testing, CI/CD, and code review. 
  • Excellent communication and cross-functional collaboration skills. 
Preferred Qualifications 
  • Experience operating InfiniBand or RDMA networking at scale. 
  • Contributions to open-source ML infrastructure projects. 
  • Familiarity with custom orchestrators or research-grade training stacks. 
  • Exposure to frontier model training operations. 
  • Experience with FinOps for AI workloads. 
How to Apply 
Would you like to know more about this opportunity? For immediate consideration, please send your resume to Jenny@bvteck.com or contact us at (908) 505-3544. Learn more about Bright Vision Technologies at www.bvteck.com.
Bright Vision Technologies is an Equal Opportunity Employer.
 

Equal Employment Opportunity (EEO) Statement

Bright Vision Technologies (BV Teck) is committed to equal employment opportunity (EEO) for all employees and applicants without regard to race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, veteran status, or any other protected status as defined by applicable federal, state, or local laws. This commitment extends to all aspects of employment, including recruitment, hiring, training, compensation, promotion, transfer, leaves of absence, termination, layoffs, and recall.

BV Teck expressly prohibits any form of workplace harassment or discrimination. Any improper interference with employees\' ability to perform their job duties may result in disciplinary action up to and including termination of employment.

Apply