ZipRecruiter

Log In

1

Machine Learning Infrastructure Engineer Jobs in San Jose, CA

previous pagenext page

Showing results 21-40

All JobsMachine Learning Infrastructure Engineer JobsMachine Learning Infrastructure Engineer Jobs in San Jose, CA

Machine Learning Infrastructure Engineer information

See San Jose, CA salary details

$54.5K

$148.9K

$213.3K

How much do machine learning infrastructure engineer jobs pay per year?

As of Aug 22, 2026, the average yearly pay for machine learning infrastructure engineer in San Jose, CA is $148,920.00, according to ZipRecruiter salary data. Most workers in this role earn between $126,000.00 and $165,200.00 per year, depending on experience, location, and employer.

What is a machine learning infrastructure engineer?

A Machine Learning Infrastructure Engineer designs, builds, and maintains the systems that support the development and deployment of machine learning models. This includes managing data pipelines, optimizing model training and inference, and ensuring scalability and reliability in production environments. They work closely with data scientists, ML engineers, and DevOps teams to create efficient workflows and infrastructure. Key technologies often include cloud platforms, containerization, orchestration tools, and distributed computing frameworks.

What are the key skills and qualifications needed to thrive as a machine learning infrastructure engineer?

To thrive as a Machine Learning Infrastructure Engineer, you need a strong background in computer science, cloud computing, distributed systems, and experience with machine learning frameworks, often supported by a degree in a related field. Familiarity with tools such as Docker, Kubernetes, Terraform, as well as cloud platforms like AWS, GCP, or Azure, and certifications in cloud or DevOps technologies are highly valued. Strong problem-solving abilities, effective communication, and collaboration skills help engineers work seamlessly with data scientists and cross-functional teams. These skills are essential to design, implement, and maintain robust, scalable infrastructure that enables efficient machine learning development and deployment.

What are some common challenges faced by machine learning infrastructure engineers, and how can these be addressed on the job?

Machine Learning Infrastructure Engineers often face challenges such as ensuring infrastructure scalability, managing resource allocation, and maintaining system reliability while supporting rapid experimentation by data science teams. Balancing the needs for flexibility in research environments with production-grade stability requires a deep understanding of both engineering best practices and the unique requirements of machine learning workflows. Collaboration with data scientists, clear communication about infrastructure capabilities, and staying current with fast-evolving technologies are key strategies for success. Most companies encourage ongoing learning and provide opportunities to contribute to architecture decisions, which makes this a rewarding environment for problem-solvers and innovators.

What are popular job titles related to Machine Learning Infrastructure Engineer jobs in San Jose, CA?

For Machine Learning Infrastructure Engineer jobs in San Jose, CA, the most frequently searched job titles are:

What job categories do people searching Machine Learning Infrastructure Engineer jobs in San Jose, CA look for?

The top searched job categories for Machine Learning Infrastructure Engineer jobs in San Jose, CA are:

What cities near San Jose, CA are hiring for Machine Learning Infrastructure Engineer jobs?

Cities near San Jose, CA with the most Machine Learning Infrastructure Engineer job openings:

Machine Learning Infrastructure Engineer jobs near you
Infographic showing various Machine Learning Infrastructure Engineer job openings in San Jose, CA as of August 2026, with employment types broken down into 1% As Needed, 76% Full Time, 21% Part Time, and 2% Contract. Highlights an 88% Physical, 2% Hybrid, and 10% Remote job distribution, with an average salary of $148,920 per year, or $71.6 per hour.

Senior Machine Learning Infrastructure Engineer

PlusAI

Santa Clara, CA • On-site, Remote

$160K - $200K/yr

Full-time

Re-posted 6 days ago

Job description

PlusAI is a Physical AI company pioneering AI-based virtual driver software for factory-built autonomous trucks. Headquartered in Silicon Valley with operations in the United States and Europe, Plus was named by Fast Company as one of the World’s Most Innovative Companies. Partners including TRATON GROUP’s Scania, MAN, and International brands, Hyundai Motor Company, Iveco Group, Bosch, and DSV are working with Plus to accelerate the deployment of next-generation autonomous trucks. If you’re ready to make a huge impact and drive the future of autonomy, Plus is looking for talented individuals to join its fast-growing teams.

As a Senior ML Infrastructure Engineer at Plus, you will design scalable architectures capable of handling petabytes of data while ensuring optimal performance for both training and inference phases. You will build robust pipelines for managing model versioning systems and experiment tracking frameworks, which are essential for maintaining reproducibility across experiments. Additionally, you will be responsible for managing large-scale GPU clusters. This role offers unparalleled opportunities—both technically and professionally—for individuals passionate about solving challenging problems using modern cloud-native technologies. Ideal candidates thrive in environments that leverage tools such as Docker containers orchestrated via Kubernetes clusters, seamlessly integrated with state-of-the-art deep learning frameworks like PyTorch or TensorFlow. If you are eager to push the boundaries of what's possible in machine learning infrastructure and contribute to cutting-edge solutions, this position is an excellent fit!
Responsibilities:
  • Design and develop scalable, high-performance systems for training, inference, deploying, and monitoring ML models at scale.
  • Build and maintain efficient data pipelines, model versioning systems, and experiment tracking frameworks.
  • Collaborate with cross-functional teams, including ML researchers and engineers, to identify bottlenecks and improve platform usability.
  • Implement distributed systems and storage solutions optimized for machine learning workloadsDrive improvements in CI/CD workflows for ML models and infrastructure.
  • Ensure high availability and reliability of the ML platform by implementing robust monitoring, logging, and alerting systems.
  • Stay current with industry trends and integrate relevant tools and frameworks to enhance the platform.
  • Mentor junior engineers and contribute to a culture of technical excellence
  • Ensure that your work is performed in accordance with the company’s Quality Management System (QMS) requirements and contribute to continuous improvement efforts.
  • Ensure team compliance with QMS, monitor quality, and drive process improvements.
Required Skills:
  • Phd or MS in Computer Science, Electrical Engineering, or related field
  • Good oral and written communication skills
  • Phd new grad or Masters with 3+ years of software engineering experience with a focus on ML infrastructure or distributed systems.
  • Proficiency in in Python, C++, SQL
  • Deep understanding of containerization, orchestration technologies, distributed ML workload, and experiment tracking tools (e.g., Docker, Kubernetes, multiprocessing, Kubeflow, and mlflow)
  • Deploy and manage resources across multiple cloud platforms (AWS, GCP, or on-prem environments)
  • Proficiency in at least one deep learning framework, such as PyTorch and data pipeline tools (e.g., Apache Airflow, Prefect).
  • Strong knowledge of distributed systems, databases, and storage solutions.
  • Extensive software design and development skills.
  • Ability to learn and adapt to new technologies and contribute in a productive environment.
Preferred Skills:
  • Familiarity with fundamental deep learning architectures, such as Convolutional Neural Networks (CNNs) and Transformer models
  • Experience in building large-scale ML datasets, MLOps pipelines, and distributed computing frameworks like Ray
  • Experience working with autonomous vehicles or robotics
 
Salary Range:
  • $160,000 - $200,000 a year
Our compensations (cash and equity) are determined based on the position, your location, qualifications, and experience.

Your opportunities joining PlusAI
Work, learn and grow in a highly future-oriented, innovative and dynamic field.
Wide range of opportunities for personal and professional development.
Catered free lunch, unlimited snacks and beverages.
Highly competitive salary and benefits package, including 401(k) plan.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.