ZipRecruiter

Log In

1

Ai Infrastructure Jobs in Raleigh, NC (NOW HIRING)

Cisco

Senior Linux Security Engineer

Durham, NC · On-site

$118K - $160K/yr

You will partner with our Prime Security Architects across infrastructure, driving security improvements across compute, storage, containers, AI, and platform services. You'll collaborate extensively ...

VAST Data

Linux Kernel Developer

Raleigh, NC · On-site

We are building the enterprise software infrastructure to capture, catalog, refine, enrich, and protect massive datasets and make them available for real-time data analysis and AI training and ...

VAST Data

Linux Kernel Developer

Raleigh, NC · On-site

We are building the enterprise software infrastructure to capture, catalog, refine, enrich, and protect massive datasets and make them available for real-time data analysis and AI training and ...

Apply Early

RELX

Senior Software Engineer

Raleigh, NC · On-site +1

$95K - $158K/yr

Are you passionate about building scalable AI infrastructure and shaping governance for cutting-edge GenAI solutions? Do you enjoy collaborating across teams while mentoring others and translating ...

New

LexisNexis

Senior Software Engineer

Raleigh, NC · On-site +1

$95K - $158K/yr

Are you passionate about building scalable AI infrastructure and shaping governance for cutting-edge GenAI solutions? Do you enjoy collaborating across teams while mentoring others and translating ...

New

VAST Data

DevOps Engineer (East Coast)

Raleigh, NC · On-site +1

$51.25 - $70.25/hr

We are building the enterprise software infrastructure to capture, catalog, refine, enrich, and protect massive datasets and make them available for real-time data analysis and AI training and ...

next page

Showing results 1-20

All JobsAi Infrastructure JobsAi Infrastructure Jobs in Raleigh, NC

Ai Infrastructure information

See Raleigh, NC salary details

$27

$57

$84

How much do ai infrastructure jobs pay per hour?

As of Jul 3, 2026, the average hourly pay for ai infrastructure in Raleigh, NC is $57.53, according to ZipRecruiter salary data. Most workers in this role earn between $46.73 and $67.07 per hour, depending on experience, location, and employer.

What is the difference between Ai Infrastructure vs Data Engineer?

AspectAi InfrastructureData Engineer
Required CredentialsBachelor's in CS, Engineering, or related; knowledge of cloud platforms and AI toolsBachelor's in CS, Data Science, or related; programming and database skills
Work EnvironmentCloud environments, AI model deployment, infrastructure setupData pipelines, database management, data processing
Employer & Industry UsageTech companies, AI startups, cloud providersTech firms, finance, healthcare, e-commerce

Ai Infrastructure professionals focus on building and maintaining the hardware and software systems that support AI models, while Data Engineers develop and manage data pipelines and databases. Both roles require technical skills and often collaborate but serve different core functions within AI and data ecosystems.

How much do AI infrastructure engineers make?

AI infrastructure engineers typically earn between $100,000 and $150,000 annually, depending on experience, location, and company size. Senior roles or those with specialized skills in cloud platforms and hardware optimization can earn higher salaries, often exceeding $180,000 per year.

What are AI infrastructure jobs?

AI infrastructure jobs involve designing, building, and maintaining the hardware, software, and network systems necessary to support artificial intelligence applications. These roles often require knowledge of cloud computing, data centers, machine learning frameworks, and system optimization to ensure reliable and efficient AI model deployment and operation.

What are the key skills and qualifications needed to thrive in AI Infrastructure, and why are they important?

To thrive in AI Infrastructure, you need expertise in software engineering, distributed systems, cloud platforms, and a solid understanding of machine learning workflows, often supported by degrees in computer science or related fields. Familiarity with tools like Kubernetes, Docker, Terraform, and cloud services (AWS, GCP, Azure), as well as experience with CI/CD pipelines and monitoring systems, is essential. Strong problem-solving abilities, effective communication, and adaptability help professionals excel in cross-functional teams and rapidly evolving environments. These skills and qualities are crucial for building scalable, reliable systems that power AI applications and support organizational innovation.

What is a $900000 AI job?

A $900,000 AI job typically refers to a high-level position in artificial intelligence, such as a senior AI researcher, machine learning director, or AI architect, often requiring advanced skills in programming, data analysis, and deep learning. These roles usually involve leadership responsibilities, extensive experience, and may include stock options or bonuses that contribute to the high total compensation.

What are common challenges faced by professionals working in AI Infrastructure roles, and how can they be addressed?

Professionals in AI Infrastructure roles often encounter challenges related to scalability, system reliability, and integration with existing IT environments. Managing rapidly growing datasets and ensuring seamless deployment of machine learning models can be complex, requiring robust automation and monitoring tools. Collaboration with data scientists, software engineers, and DevOps teams is critical to ensure infrastructure meets the evolving needs of AI projects. Staying updated with the latest cloud technologies and best practices can help address these challenges and drive successful AI implementations.

What is AI Infrastructure?

AI infrastructure refers to the combination of hardware, software, and cloud-based solutions that support the development, deployment, and scaling of artificial intelligence applications. It includes components such as GPUs, CPUs, storage systems, networking, data management tools, and machine learning frameworks. The goal of AI infrastructure is to provide the computational power and resources needed to train, test, and run AI models efficiently, whether on-premises or in the cloud. Organizations invest in robust AI infrastructure to accelerate innovation, manage large datasets, and ensure the reliability of their AI systems.

What engineer makes $500,000 a year?

Senior AI infrastructure engineers or machine learning engineers with extensive experience, specialized skills, and advanced certifications can earn salaries approaching or exceeding $500,000 annually, especially in high-cost-of-living areas or large tech companies. These roles often require expertise in cloud platforms, distributed systems, and deep learning frameworks, along with leadership responsibilities and a strong track record of impactful projects.
What are popular job titles related to Ai Infrastructure jobs in Raleigh, NC? For Ai Infrastructure jobs in Raleigh, NC, the most frequently searched job titles are:
What job categories do people searching Ai Infrastructure jobs in Raleigh, NC look for? The top searched job categories for Ai Infrastructure jobs in Raleigh, NC are:
What cities near Raleigh, NC are hiring for Ai Infrastructure jobs? Cities near Raleigh, NC with the most Ai Infrastructure job openings:
Ai Infrastructure jobs near you
Infographic showing various Ai Infrastructure job openings in Raleigh, NC as of June 2026, with employment types broken down into 70% Full Time, 28% Part Time, and 2% Contract. Highlights an 66% Physical, 3% Hybrid, and 31% Remote job distribution, with an average salary of $119,666 per year, or $57.5 per hour.
Senior Solutions Architect - AI Factory Deployment

Senior Solutions Architect - AI Factory Deployment

Nvidia

Durham, NC • On-site

Full-time

Posted 5 days ago

Job description

We are seeking an ambitious Senior Solutions Architect - AI Factory Deployment to join our NVIDIA Infrastructure Specialists team in Santa Clara! This role is uniquely positioned to develop, deploy, and validate AI factories end to end. You will focus on running and debugging AI/LLM workloads and benchmarks on Linux-based GPU clusters, using NCCL and collectives like AllReduce and AllToAll to improve performance and scalability.

As part of our world-class team, you will bring to bear observability and automation to improve benchmarks and validation. You will serve as the expert when workloads or benchmarks do not perform flawlessly. You will collaborate across NVIDIA to ensure AI factories are prepared for customers, validating hardware and software for modern AI deployments.

What You Will be Doing:

  • Set up, adjust, and verify AI factory environments across multi-GPU and multi-node Linux clusters.

  • Ensure configurations align with guidelines for NCCL, collectives, and distributed training frameworks.

  • Own the execution of key AI/LLM benchmarks, including setup, orchestration, result collection, and analysis.

  • Investigate and resolve issues when training jobs or benchmarks fail, hang, or underperform.

  • Build and improve observability for AI factories (metrics, logs, traces, dashboards) to understand workload behavior and system health.

  • Develop automation (Python, Shell) for running benchmarks, collecting results, and performing regression checks

  • Examine communication patterns and NCCL usage for AI/LLM workloads, concentrating on collectives such as AllReduce and AllToAll.

  • Recommend changes to job configuration, parallelism strategies, and cluster settings to improve throughput, latency, and scaling efficiency.

  • Work closely with hardware, software, networking, datacenter, and product teams to prepare AI factories for customer use.

  • Contribute to documentation, guidelines, and readiness collateral that support internal collaborators and customer-facing teams.

What We Need to See:

  • Bachelor's degree or equivalent experience in Computer Science, Mathematics, Engineering, Physics, or related field.

  • More than 6+ years of experience managing Linux-based systems in HPC, distributed systems, or extensive AI/ML settings.

  • Hands-on experience running AI/ML workloads on multi-GPU and/or multi-node clusters, with practical knowledge of NCCL.

  • Solid grasp of collective communication patterns, particularly AllReduce and AllToAll, and how they are applied in contemporary ML/LLM training.

  • Familiarity with LLM training and/or inference workflows using frameworks such as PyTorch or TensorFlow.

  • Proficiency with Python and Shell/Bash for scripting, automation, and tooling.

  • Experience with benchmarking (crafting, executing, and interpreting performance benchmarks).

  • Comfortable working with observability data (metrics, logs, dashboards) to troubleshoot and optimize complex distributed workloads.

  • Strong communication skills and the ability to work effectively with cross-functional teams.

Ways to Stand Out From the Crowd:

  • Experience with AI factory or large-scale AI infrastructure build, deployment, or operations.

  • Background in HPC performance engineering, SRE, or systems performance analysis for GPU-accelerated environments.

  • Familiarity with observability stacks (e.g., metrics/monitoring, logging, tracing systems) used for large distributed systems.

  • Experience building automation and CI-style pipelines for running and validating benchmarks at scale.

  • Demonstrated desire to use AI to solve practical problems, improve workflows, and guide data-driven decisions.

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD for Level 4, and 224,000 USD - 356,500 USD for Level 5.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until May 3, 2026.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
Nvidia logo

About Nvidia

Sourced by ZipRecruiter

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It's a unique legacy of innovation that's fueled by great technology--and amazing people. Today, we're tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what's never been done before takes vision, innovation, and the world's best talent.

Industry

Computer and electronic product manufacturing

Company size

10,000+ Employees

Headquarters location

Santa Clara, CA, US

Year founded

1993

View All Nvidia Jobs

Apply