ZipRecruiter

Log In

1

Senior Ai Infrastructure Engineer Jobs (NOW HIRING)

Together AI

AI Infrastructure Engineer

San Francisco, CA · On-site

$190K - $270K/yr

As an AI Infrastructure Engineer at Together, you are responsible for keeping all user-facing services and production systems running smoothly. You are a blend of a pragmatic operator and a software ...

BNY

AI Infrastructure Engineer

Lake Mary, FL · On-site

$94K - $123K/yr

They are seeking an AI Infrastructure Engineer to support enterprise-grade NVIDIA AI infrastructure, automate operations, and maintain AI training and inference pipelines. Responsibilities : • Be ...

BNY

AI Infrastructure Engineer

Lake Mary, FL · On-site

$94K - $123K/yr

They are seeking an AI Infrastructure Engineer to support enterprise-grade NVIDIA AI infrastructure, automate operations, and maintain AI training and inference pipelines. Responsibilities : • Be ...

Autonomous Solutions

AI Infrastructure Engineer IV

Lehi, UT · On-site

$100K - $132K/yr

As an AI Infrastructure Engineer IV , you will play a critical role in designing, building, and maintaining the systems that power our AI and machine learning capabilities. You will ensure our ...

Apply Early

Radiology Partners

Senior AI Infrastructure Engineer

$111K - $151K/yr

RADIOLOGY PARTNERS OVERVIEW Radiology Partners, through its owned and affiliated practices, is a leading radiology practice in the U.S., serving hospitals and other healthcare facilities across the ...

Utilidata

AI Infrastructure Engineer

Ann Arbor, MI · Remote

$170K - $210K/yr

The AI Infrastructure Engineer is responsible for designing, building, and owning the end-to-end infrastructure that serves Utilidata's AI and ML models across edge deployments, cloud environments ...

Utilidata

AI Infrastructure Engineer

Ann Arbor, MI · On-site +1

$170K - $210K/yr

The AI Infrastructure Engineer is responsible for designing, building, and owning the end-to-end infrastructure that serves Utilidata's AI and ML models across edge deployments, cloud environments ...

BNY

AI Infrastructure Engineer

Lake Mary, FL · Hybrid

$94K - $123K/yr

AI Infrastructure Engineer At BNY, our culture allows us to run our company better and enables employees' growth and success. As a leading global financial services company at the heart of the global ...

BNY

AI Infrastructure Engineer

Lake Mary, FL · Hybrid

$94K - $123K/yr

AI Infrastructure Engineer At BNY, our culture allows us to run our company better and enables employees' growth and success. As a leading global financial services company at the heart of the global ...

Autonomous Solutions

AI Infrastructure Engineer IV

Mendon, UT · On-site

$93K - $122K/yr

As an AI Infrastructure Engineer IV , you will play a critical role in designing, building, and maintaining the systems that power our AI and machine learning capabilities. You will ensure our ...

Apply Early

Percepta

AI Infrastructure Engineer

$110K - $144K/yr

They are seeking an AI Infrastructure Engineer to own the infrastructure and operational reliability that powers their AI systems, focusing on defining infrastructure patterns and building ...

BNY Mellon

AI Infrastructure Engineer

Lake Mary, FL · On-site

$94K - $123K/yr

AI Infrastructure Engineer At BNY, our culture allows us to run our company better and enables employees' growth and success. As a leading global financial services company at the heart of the global ...

next page

Showing results 1-20

All JobsSenior Ai Infrastructure Engineer Jobs

Senior Ai Infrastructure Engineer information

See salary details

$22.5K

$127K

$175.5K

How much do senior ai infrastructure engineer jobs pay per year?

As of Jul 4, 2026, the average yearly pay for senior ai infrastructure engineer in the United States is $126,969.00, according to ZipRecruiter salary data. Most workers in this role earn between $108,500.00 and $147,500.00 per year, depending on experience, location, and employer.

What does a Senior AI Infrastructure Engineer do?

A Senior AI Infrastructure Engineer is responsible for designing, building, and maintaining the large-scale computing systems that support artificial intelligence (AI) and machine learning (ML) workloads. They work on optimizing data pipelines, managing cloud or on-premise infrastructure, ensuring scalability, and enabling efficient training and deployment of AI models. These professionals collaborate closely with data scientists, software engineers, and IT teams to create robust, high-performance environments that support the rapid development and deployment of AI solutions.

What are some typical challenges faced by Senior AI Infrastructure Engineers when scaling AI systems for production?

Senior AI Infrastructure Engineers often encounter challenges related to managing large-scale data pipelines, ensuring low-latency model serving, and maintaining system reliability as user demand grows. Balancing resource allocation for compute-intensive workloads, optimizing infrastructure costs, and implementing robust monitoring are common hurdles. Collaboration with data scientists, DevOps, and product teams is crucial to streamline deployment cycles and rapidly address issues as they arise. Mastery of distributed systems and cloud platforms often distinguishes top performers in this role.

What are the key skills and qualifications needed to thrive as a Senior AI Infrastructure Engineer, and why are they important?

To thrive as a Senior AI Infrastructure Engineer, you need deep expertise in computer science, cloud computing, distributed systems, and AI/ML frameworks, often supported by a relevant degree and significant experience. Proficiency with tools such as Kubernetes, Docker, TensorFlow, PyTorch, and cloud platforms like AWS or Azure—as well as experience with CI/CD pipelines—is typically required. Strong problem-solving abilities, collaboration, and effective communication are standout soft skills for this role. These competencies are crucial for designing scalable, reliable AI infrastructure that supports complex machine learning workflows and organizational goals.
More about Senior Ai Infrastructure Engineer jobs
What cities are hiring for Senior Ai Infrastructure Engineer jobs? Cities with the most Senior Ai Infrastructure Engineer job openings:
What are the most commonly searched types of Ai Infrastructure Engineer jobs? The most popular types of Ai Infrastructure Engineer jobs are:
What states have the most Senior Ai Infrastructure Engineer jobs? States with the most job openings for Senior Ai Infrastructure Engineer jobs include:
What job categories do people searching Senior Ai Infrastructure Engineer jobs look for? The top searched job categories for Senior Ai Infrastructure Engineer jobs are:
Senior Ai Infrastructure Engineer jobs near you
Infographic showing various Senior Ai Infrastructure Engineer job openings in the United States as of June 2026, with employment types broken down into 96% Full Time, 2% Part Time, and 2% Contract. Highlights an 66% Physical, 3% Hybrid, and 31% Remote job distribution, with an average salary of $126,969 per year, or $61 per hour.
AI Infrastructure Engineer

AI Infrastructure Engineer

Together AI

San Francisco, CA • On-site

$190K - $270K/yr

Full-time

Medical

Posted 6 days ago

Job description

As an AI Infrastructure Engineer at Together, you are responsible for keeping all user-facing services and production systems running smoothly. You are a blend of a pragmatic operator and a software engineer that applies sound engineering principles, operational discipline, and mature automation to our operating environments and codebase.
You specialize in systems (operating systems, storage subsystems, networking), while implementing best practices for availability, reliability and scalability, with varied interests in algorithms and distributed systems.
Responsibilities
  • Participate in on-call rotation (Pagerduty) to respond to production incidents
  • Build and run our infrastructure with Ansible, Terraform, and Kubernetes to enable scaling to a massive number of concurrent users
  • Build monitoring systems to ensure the highest quality service for our customers
  • Design and implement operational processes (such as deployments and upgrades)
  • Debug production issues across all services and levels of the stack
  • Identify improvements for the product architecture from the reliability, performance and availability perspectives
  • Plan the growth of Together AI's infrastructure

Requirements
  • 5+ years of professional AI Infra or related experience
  • Bachelor's degree in Computer Science or a related field or equivalent work experience
  • Knowledge of Ansible (roles, playbooks), Terraform, and Kubernetes
  • Proficiency in programming/scripting languages
  • Direct experience in monitoring and observability practices
  • Knowledge of cloud services
  • Ability to thrive in a collaborative environment involving different stakeholders and subject matter experts

About Together AI
Together AI is a research-driven artificial intelligence company. We believe open and transparent AI systems will drive innovation and create the best outcomes for society, and together we are on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets to advance the frontier of AI, and our team has been behind technological advancement such as FlashAttention, Hyena, FlexGen, and RedPajama. We invite you to join a passionate group of researchers and engineers in our journey in building the next generation AI infrastructure.
Compensation
We offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full-time position is: $190,000 - $270,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.
Equal Opportunity
Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Apply