ZipRecruiter

Log In

1

Machine Learning Infrastructure Engineer Jobs in San Jose, CA

next page

Showing results 1-20

All JobsMachine Learning Infrastructure Engineer JobsMachine Learning Infrastructure Engineer Jobs in San Jose, CA

Machine Learning Infrastructure Engineer information

See San Jose, CA salary details

$54.5K

$148.9K

$213.3K

How much do machine learning infrastructure engineer jobs pay per year?

As of Jul 27, 2026, the average yearly pay for machine learning infrastructure engineer in San Jose, CA is $148,920.00, according to ZipRecruiter salary data. Most workers in this role earn between $126,000.00 and $165,200.00 per year, depending on experience, location, and employer.

What are some common challenges faced by Machine Learning Infrastructure Engineers, and how can these be addressed on the job?

Machine Learning Infrastructure Engineers often face challenges such as ensuring infrastructure scalability, managing resource allocation, and maintaining system reliability while supporting rapid experimentation by data science teams. Balancing the needs for flexibility in research environments with production-grade stability requires a deep understanding of both engineering best practices and the unique requirements of machine learning workflows. Collaboration with data scientists, clear communication about infrastructure capabilities, and staying current with fast-evolving technologies are key strategies for success. Most companies encourage ongoing learning and provide opportunities to contribute to architecture decisions, which makes this a rewarding environment for problem-solvers and innovators.

What are the key skills and qualifications needed to thrive in the Machine Learning Infrastructure Engineer position, and why are they important?

To thrive as a Machine Learning Infrastructure Engineer, you need a strong background in computer science, cloud computing, distributed systems, and experience with machine learning frameworks, often supported by a degree in a related field. Familiarity with tools such as Docker, Kubernetes, Terraform, as well as cloud platforms like AWS, GCP, or Azure, and certifications in cloud or DevOps technologies are highly valued. Strong problem-solving abilities, effective communication, and collaboration skills help engineers work seamlessly with data scientists and cross-functional teams. These skills are essential to design, implement, and maintain robust, scalable infrastructure that enables efficient machine learning development and deployment.

What is a Machine Learning Infrastructure Engineer job?

A Machine Learning Infrastructure Engineer designs, builds, and maintains the systems that support the development and deployment of machine learning models. This includes managing data pipelines, optimizing model training and inference, and ensuring scalability and reliability in production environments. They work closely with data scientists, ML engineers, and DevOps teams to create efficient workflows and infrastructure. Key technologies often include cloud platforms, containerization, orchestration tools, and distributed computing frameworks.

What are popular job titles related to Machine Learning Infrastructure Engineer jobs in San Jose, CA? For Machine Learning Infrastructure Engineer jobs in San Jose, CA, the most frequently searched job titles are:
What job categories do people searching Machine Learning Infrastructure Engineer jobs in San Jose, CA look for? The top searched job categories for Machine Learning Infrastructure Engineer jobs in San Jose, CA are:
What cities near San Jose, CA are hiring for Machine Learning Infrastructure Engineer jobs? Cities near San Jose, CA with the most Machine Learning Infrastructure Engineer job openings:
Machine Learning Infrastructure Engineer jobs near you
Infographic showing various Machine Learning Infrastructure Engineer job openings in San Jose, CA as of July 2026, with employment types broken down into 94% Full Time, 3% Part Time, and 3% Contract. Highlights an 88% Physical, 5% Hybrid, and 7% Remote job distribution, with an average salary of $148,920 per year, or $71.6 per hour.

Machine Learning Infrastructure Engineer

Bright Vision Technologies

Sunnyvale, CA • Remote

$100K - $150K/yr

Full-time

Posted 3 days ago

Job description

Machine Learning Infrastructure Engineer – Remote
Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States.
This is a fantastic opportunity to join an established and well-respected organization offering tremendous career growth potential.
Job Title: Machine Learning Infrastructure Engineer
Location: 100% Remote (U.S.)
Position Type: Full-time, Direct W2
Salary Range: $100,000–$150,000 Annually
Experience Required: 6+ years
Sponsorship: U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates are encouraged to apply. We are unable to sponsor new H-1B visa petitions for this position.
Job Summary
We are seeking a Machine Learning Infrastructure Engineer to design, build, and operate high-performance, highly reliable inference platforms for serving large machine learning models in production. The role focuses on the systems engineering side of AI deployment, including request routing, batching, caching, autoscaling, GPU utilization, and end-to-end observability across diverse model workloads. The ideal candidate brings strong distributed systems and performance engineering expertise, has shipped serving systems at scale, and understands the trade-offs between latency, throughput, cost, and quality in ML serving.
Required Qualifications
  • Bachelor’s or Master’s degree in Computer Science or a related field.
  • Six or more years of experience in distributed systems, infrastructure, or ML platform engineering.
  • Strong proficiency in Python and a systems language such as Go, Rust, or C++.
  • Deep experience operating high-throughput, low-latency services in production.
  • Hands-on experience with LLM or large model inference frameworks such as vLLM or TensorRT-LLM.
  • Strong understanding of GPU architecture, memory hierarchies, and accelerator utilization.
  • Familiarity with Kubernetes, autoscaling, and modern cloud platforms.
  • Experience with observability stacks including metrics, tracing, and structured logging.
  • Solid grounding in performance engineering and capacity planning.
  • Strong communication and incident response skills.
Preferred Qualifications
  • Open-source contributions to model serving infrastructure.
  • Experience with multi-region or globally distributed AI serving.
  • Familiarity with model quantization, distillation, and compression techniques.
  • Exposure to FinOps for AI workloads and cost-efficient serving design.
  • Experience supporting external-facing AI APIs at scale.
How to Apply
Would you like to know more about this opportunity? For immediate consideration, please send your resume to hilda@bvteck.com.
Bright Vision Technologies is an Equal Opportunity Employer.

Equal Employment Opportunity (EEO) Statement

Bright Vision Technologies (BV Teck) is committed to equal employment opportunity (EEO) for all employees and applicants without regard to race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, veteran status, or any other protected status as defined by applicable federal, state, or local laws. This commitment extends to all aspects of employment, including recruitment, hiring, training, compensation, promotion, transfer, leaves of absence, termination, layoffs, and recall.

BV Teck expressly prohibits any form of workplace harassment or discrimination. Any improper interference with employees\' ability to perform their job duties may result in disciplinary action up to and including termination of employment.

Apply