ZipRecruiter

Log In

1

Ml Infrastructure Engineer Jobs in Raleigh, NC (NOW HIRING)

NVIDIA

Senior ML Platform Engineer

Durham, NC · On-site

$101K - $138K/yr

They are seeking a Senior ML Platform Engineer to architect, build, and scale high-performance ML ... Responsibilities : • Design, build, and maintain our core ML platform infrastructure as code ...

Wells Fargo

Lead Infrastructure Engineer

Raleigh, NC

$104K - $137K/yr

Wells Fargo is seeking a Lead Infrastructure Engineer within the Core Infrastructure Services Engineering team to deliver services that provide AI & ML capabilities across multiple clouds. In this ...

Google

Software Engineer AI/ML Networking

Durham, NC · On-site

They are seeking a Software Engineer specializing in AI/ML Networking to work on critical projects, focusing on hardware-software solutions and machine learning infrastructure. Responsibilities : • ...

Nvidia

Senior ML Platform Engineer

Durham, NC

$101K - $138K/yr

In this role, you will architect, build, and scale our high-performance ML infrastructure using ... Join our top team and apply your SRE and software engineering skills to craft robust, user-friendly ...

Google

Software Engineer AI/ML Networking

Raleigh, NC · On-site

... ML infrastructure (e.g., model deployment, model evaluation, optimization, data processing ... Experience with Machine Learning Infrastructure. About the job Google's software engineers develop ...

Samprasoft

Machine Learning Engineer

Durham, NC

Engineering, Computer Science, etc.) * 8+ years of proven experience in implementing Big data solutions in data analytics space * 2+ years of experience in developing ML infrastructure and MLOps in ...

OptiTrack

Machine Learning & Operations Engineer

Durham, NC · Remote

$71K - $96K/yr

Design and maintain automated ML training pipelines. * Build infrastructure for large-scale ... More typical DevOps responsibilities for software development as required. Requirements Required ...

OptiTrack

Machine Learning & Operations Engineer

Durham, NC · Remote

$67K - $90K/yr

Design and maintain automated ML training pipelines. * Build infrastructure for large-scale ... More typical DevOps responsibilities for software development as required. Requirements Required ...

next page

Showing results 1-20

All JobsMl Infrastructure Engineer JobsMl Infrastructure Engineer Jobs in Raleigh, NC

Ml Infrastructure Engineer information

See Raleigh, NC salary details

$45.2K

$123.5K

$176.9K

How much do ml infrastructure engineer jobs pay per year?

As of Jun 11, 2026, the average yearly pay for ml infrastructure engineer in Raleigh, NC is $123,519.00, according to ZipRecruiter salary data. Most workers in this role earn between $104,500.00 and $137,100.00 per year, depending on experience, location, and employer.

What is the difference between Ml Infrastructure Engineer vs Data Engineer?

AspectML Infrastructure EngineerData Engineer
Required CredentialsBachelor's/Master's in CS, experience with cloud platforms, scripting, and ML toolsBachelor's/Master's in CS, experience with databases, ETL, and data pipelines
Work EnvironmentFocus on deploying and maintaining ML systems, cloud infrastructure, and automationDesigning and building data pipelines, managing large datasets, and data storage
Employer & Industry UsageTech companies, AI startups, research labsFinance, healthcare, e-commerce, and data-driven industries

The ML Infrastructure Engineer specializes in building and maintaining the infrastructure that supports machine learning models, focusing on deployment, scalability, and automation. In contrast, Data Engineers primarily develop data pipelines and manage large datasets to enable data analysis and business intelligence. Both roles require strong technical skills and often overlap, but their core focus areas differ significantly.

What are popular job titles related to Ml Infrastructure Engineer jobs in Raleigh, NC? For Ml Infrastructure Engineer jobs in Raleigh, NC, the most frequently searched job titles are:
What job categories do people searching Ml Infrastructure Engineer jobs in Raleigh, NC look for? The top searched job categories for Ml Infrastructure Engineer jobs in Raleigh, NC are:
What cities near Raleigh, NC are hiring for Ml Infrastructure Engineer jobs? Cities near Raleigh, NC with the most Ml Infrastructure Engineer job openings:
Ml Infrastructure Engineer jobs near you

Senior Kubernetes Platform Engineer - AI/ML Infrastructure

Webex Events (formerly Socio)

Durham, NC

$98K - $133K/yr

Other

Posted 27 days ago

Job description

Senior Kubernetes Platform Engineer - AI/ML Infrastructure

Join our Platform Engineering team to design, build, and operate large-scale, on-prem Kubernetes infrastructure powering next-generation AI/ML platforms, including GPU-enabled environments for both traditional ML and state-of-the-art LLM workloads.

You will be pivotal in defining and evolving a highly scalable Kubernetes platform that serves as the foundation for AI/ML workloads. This role combines deep Kubernetes platform engineering with AI/ML infrastructure enablement, ensuring performance, reliability, and scalability across distributed systems.

You will lead technical direction across Kubernetes control plane operations, cluster lifecycle management, and platform extensibility, while working closely with data scientists, ML engineers, and infrastructure teams to support production AI workloads at scale.

This is a senior individual contributor role focused on platform ownership, engineering excellence, and driving reliability and automation across complex distributed environments.

Your Impact / Core Responsibilities
  • Architect, build, and operate large-scale on-prem Kubernetes platforms (OpenShift/Anthos), including control plane and etcd lifecycle management
  • Define and evolve scalable, multi-tenant platform architecture supporting AI/ML and GPU-based workloads
  • Enable and optimize ML workloads including training, inference, and LLM deployment pipelines on Kubernetes
  • Build platform extensions using Kubernetes controllers, operators, CRDs, and Golang-based services
  • Implement Infrastructure as Code and automation to improve scalability, consistency, and operational efficiency
  • Drive AIOps capabilities using telemetry, automation, anomaly detection, and self-healing systems for platform reliability
  • Improve observability (metrics, logs, traces) and optimize resource utilization, scheduling, and cluster performance
  • Partner with ML engineers and data scientists to operationalize ML workflows and ensure platform readiness for AI workloads
  • Participate in on-call rotations, owning incident response, reliability, and continuous operational improvement
  • Mentor engineers and contribute to defining platform engineering standards and best practices
Minimum Qualifications
  • 8+ years of software engineering experience
  • 4+ years of hands-on Kubernetes production experience with control plane ownership
  • Strong experience operating on-prem or self-managed Kubernetes environments
  • Deep expertise in etcd management (backup, restore, recovery, upgrades)
  • Strong proficiency in Go with experience building Kubernetes controllers, operators, CRDs, and webhooks
  • Deep understanding of Kubernetes internals (API server, scheduler, controller loops, reconciliation)
  • Experience supporting AI/ML or GPU-based workloads on Kubernetes platforms
  • Proven experience operating and debugging large-scale distributed systems
  • Experience participating in on-call rotations and production incident management
Preferred Qualifications
  • Experience with bare-metal or enterprise on-prem infrastructure at scale
  • Exposure to AI/ML platforms and tooling (e.g., Kubeflow, MLflow, distributed training systems)
  • Experience building internal developer platforms or platform-as-a-service (PaaS) systems
  • Familiarity with AIOps concepts such as automated remediation and predictive operations
  • Experience applying data-driven or ML-based techniques for system reliability or capacity planning
  • Contributions to Kubernetes, CNCF, or other open-source ecosystems

Apply