ZipRecruiter

Log In

1

Ai Infrastructure Job Jobs (NOW HIRING)

Palona AI

AI Infrastructure Engineer

Los Altos, CA · On-site

$127K - $167K/yr

Palona's AI agents operate continuously in production, handle real-time guest interactions, integrate with restaurant systems, and face sharp traffic peaks. Infrastructure is therefore part of the ...

The Josef Group

AI Infrastructure Engineer

Chantilly, VA

$110K - $144K/yr

AI Infrastructure Engineer Top Secret or TS/SCI is required to start $200K to $250K Chantilly, VA What You'll Do * Deploy and optimize self-hosted LLM inference servers (vLLM, Ollama, and similar)

Palona AI

AI Infrastructure Engineer

Los Altos, CA · On-site

$150 - $230/hr

Palona's AI agents operate continuously in production, handle real-time guest interactions, integrate with restaurant systems, and face sharp traffic peaks. Infrastructure is therefore part of the ...

Palona AI

AI Infrastructure Engineer

Los Altos, CA

$127K - $167K/yr

Palona's AI agents operate continuously in production, handle real-time guest interactions, integrate with restaurant systems, and face sharp traffic peaks. Infrastructure is therefore part of the ...

Hedge Fund

AI Infrastructure Engineer

Manhattan, NY · On-site

$150 - $230/hr

Multi-Strategy / Multi-Manager hedge fund is seeking an AI Platform Engineer to design, build, and scale the foundational infrastructure that enables AI-driven development across the organization.

New

Palona AI

AI Infrastructure Engineer

New York, NY · On-site

$117K - $154K/yr

Palona's AI agents operate continuously in production, handle real-time guest interactions, integrate with restaurant systems, and face sharp traffic peaks. Infrastructure is therefore part of the ...

Palona AI

AI Infrastructure Engineer

New York, NY

$117K - $154K/yr

Palona's AI agents operate continuously in production, handle real-time guest interactions, integrate with restaurant systems, and face sharp traffic peaks. Infrastructure is therefore part of the ...

The Josef Group

AI Infrastructure Engineer

Chantilly, VA · On-site

$200 - $250/hr

AI Infrastructure Engineer Top Secret or TS/SCI is required to start $200K to $250K Chantilly, VA What You'll Do * Deploy and optimize self-hosted LLM inference servers (vLLM, Ollama, and similar)

Palona AI

AI Infrastructure Engineer

Los Altos, CA · On-site

$127K - $167K/yr

Palona's AI agents operate continuously in production, handle real-time guest interactions, integrate with restaurant systems, and face sharp traffic peaks. Infrastructure is therefore part of the ...

Utilidata

AI Infrastructure Engineer

Ann Arbor, MI · Remote

$170K - $210K/yr

The AI Infrastructure Engineer is responsible for designing, building, and owning the end-to-end infrastructure that serves Utilidata's AI and ML models across edge deployments, cloud environments ...

Axon

AI Infrastructure Engineer

Boston, MA · On-site

$116K - $153K/yr

AI Infrastructure Engineer, Corporate AI Team Team & Role Overview Axon's Corporate AI Team sits within Business Technology and builds internal-facing AI solutions that help employees reduce manual ...

Heaven-Hill

AI Infrastructure Analyst

Louisville, TN · On-site

$75 - $105/hr

AI Infrastructure Analyst G&A Analyst Louisville, KY, US This is an office based position in Louisville, KY. What the Role Is The AI Infrastructure Analyst is responsible for the day-to-day ...

New

BigBear.ai

Software Engineer (AI Infrastructure)

Columbia, MD · On-site

$170K - $201K/yr

Overview BigBear.ai is seeking a S oftware Engineer to support our AI infrastructure team. In this role, you'll help build and maintain the platform that provides the foundation for the customer's AI ...

next page

Showing results 1-20

All JobsTechnology JobsSoftware JobsAi Infrastructure Job Jobs

Ai Infrastructure Job information

See salary details

$28

$59

$87

How much do ai infrastructure job jobs pay per hour?

As of Aug 21, 2026, the average hourly pay for ai infrastructure job in the United States is $59.18, according to ZipRecruiter salary data. Most workers in this role earn between $48.08 and $68.99 per hour, depending on experience, location, and employer.

What is an AI infrastructure job?

An AI Infrastructure job involves designing, building, and maintaining the foundational systems and tools required to support artificial intelligence and machine learning workloads. These professionals work on scalable computing environments, manage large datasets, and ensure high-performance computing resources are available for AI development and deployment. Their responsibilities often include optimizing hardware (like GPUs and TPUs), developing cloud-based solutions, and establishing reliable pipelines for data and model management. AI Infrastructure specialists collaborate with data scientists, ML engineers, and IT teams to deliver robust, efficient, and secure platforms that power AI applications.

What are some common challenges faced by professionals working in AI infrastructure roles?

Professionals in AI infrastructure roles often encounter challenges such as ensuring scalability and reliability of systems to handle large volumes of data and compute-intensive workloads. Managing the integration of diverse tools and frameworks, optimizing hardware and cloud resources, and maintaining security and compliance are also critical aspects. Additionally, effective collaboration with data scientists, software engineers, and DevOps teams is essential to streamline AI model deployment and monitoring. Staying current with rapidly evolving technologies is key to overcoming these challenges.

What are the key skills and qualifications needed to thrive in an AI infrastructure role, and why are they important?

To excel in an AI Infrastructure role, you need a strong background in computer science, cloud computing, and distributed systems, often supported by a relevant degree and experience with large-scale data environments. Proficiency with tools like Kubernetes, Docker, TensorFlow, PyTorch, and cloud platforms such as AWS, Azure, or Google Cloud is essential, along with familiarity with CI/CD pipelines and automation frameworks. Strong problem-solving, collaboration, and communication skills help drive innovation and facilitate cross-functional teamwork. These abilities ensure the robust, scalable, and efficient deployment of AI solutions that meet organizational needs.

What is the difference between Ai Infrastructure Job vs Data Engineer?

AspectAi Infrastructure JobData Engineer
Required CredentialsBachelor's in CS, Engineering, or related; knowledge of AI frameworksBachelor's in CS, Data Science, or related; programming skills in SQL, Python
Work EnvironmentData centers, cloud platforms, AI labsData pipelines, cloud environments, database systems
Industry UsageAI development, machine learning deploymentData processing, analytics, data pipeline creation

Ai Infrastructure Jobs focus on building and maintaining the hardware and software systems that support AI applications, including cloud and data center infrastructure. Data Engineers primarily develop and manage data pipelines and storage solutions to enable data analysis and machine learning. While both roles require technical skills and work in tech environments, Ai Infrastructure Jobs are more hardware and system-focused, whereas Data Engineers concentrate on data flow and management.

More about Ai Infrastructure Job jobs

What cities are hiring for Ai Infrastructure Job jobs?

Cities with the most Ai Infrastructure Job job openings:

What states have the most Ai Infrastructure Job jobs?

States with the most job openings for Ai Infrastructure Job jobs include:

What job categories do people searching Ai Infrastructure Job jobs look for?

The top searched job categories for Ai Infrastructure Job jobs are:

Ai Infrastructure Job jobs near you
Infographic showing various Ai Infrastructure Job job openings in the United States as of August 2026, with employment types broken down into 1% As Needed, 70% Full Time, 23% Part Time, and 6% Contract. Highlights an 95% Physical, 1% Hybrid, and 4% Remote job distribution, with an average salary of $123,103 per year, or $59.2 per hour.

AI Infrastructure Systems Engineer

Together AI

San Francisco, CA • On-site

$190K - $270K/yr

Full-time

Medical

Re-posted 7 days ago

Job description

Build the infrastructure powering the next generation of AI.
At Together AI, you'll build and operate one of the world's largest GPU fleets used for frontier model training and inference. This isn't a traditional infrastructure role-we're looking for engineers who love building systems, automating everything, and solving problems at massive scale.
You'll Thrive Here If You:
  • Love building systems that replace repetitive operational work.
  • Think of infrastructure as a software engineering problem.
  • Enjoy solving hard problems with no existing playbook.
  • Care deeply about performance, reliability, and scale.
  • Want to build technology that powers frontier AI models.

Our mission is simple: build AI infrastructure that largely runs itself-where intelligent systems deploy, monitor, diagnose, optimize, and heal GPU fleets at massive scale. Every system you build will directly improve the speed, efficiency, and reliability of one of the world's most advanced AI compute platforms.
If you enjoy writing software more than clicking dashboards, obsess over eliminating manual work, and want to build infrastructure that manages tens of thousands of GPUs autonomously, we'd love to talk.
Responsibilities
  • Design and build fleet automation systems that provision, validate, deploy, upgrade, repair, and retire GPU clusters with minimal human intervention.
  • Build AI Infrastructure Agents that automate deployment, root-cause failures, incident triage, and autonomous remediation.
  • Develop Fleet Intelligence platforms that continuously monitor hardware health, firmware, networking, storage, thermals, and workload performance to predict failures before they impact customers.
  • Build software that maximizes GPU availability, utilization, performance, and reliability across thousands of accelerators.
  • Create automated validation systems for GPUs, InfiniBand/RoCE fabrics, NVLink/NVSwitch, storage, and distributed AI workloads.
  • Build internal platforms and developer tools that allow infrastructure to be managed through software-not manual operations.
  • Continuously improve deployment velocity, reliability, and operational efficiency through automation.
  • Partner closely with hardware, networking, platform, and AI teams to push the limits of AI infrastructure.
Requirements
  • 3+ years building distributed systems, infrastructure platforms, or large-scale backend software.
  • Strong software engineering skills in Python, Go, or Rust.
  • Experience building platforms, automation systems, or developer infrastructure.
  • Experience with Linux, Kubernetes, Terraform, Ansible, or similar infrastructure technologies.
  • Strong systems thinking with the ability to understand problems across hardware and software.
  • A passion for solving complex infrastructure challenges through software.
  • An automation-first mindset-if a task is repeated, your instinct is to build a system to eliminate it.
Bonus Experience
  • GPU infrastructure, CUDA, NCCL, NVLink/NVSwitch
  • InfiniBand or RoCE networking
  • Bare-metal provisioning and lifecycle management
  • Large-scale AI training or inference clusters
  • Hardware health monitoring and predictive failure detection
  • Distributed storage systems
  • AI agents and autonomous infrastructure operations
About Together AI
Together AI is a research-driven artificial intelligence company. We believe open and transparent AI systems will drive innovation and create the best outcomes for society, and together we are on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets to advance the frontier of AI, and our team has been behind technological advancement such as FlashAttention, Hyena, FlexGen, and RedPajama. We invite you to join a passionate group of researchers and engineers in our journey in building the next generation AI infrastructure.
Compensation
We offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full-time position is: $190,000 - $270,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.
Equal Opportunity
Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Apply