ZipRecruiter

Log In

1

Ai Hardware Engineer Jobs in Raleigh, NC (NOW HIRING)

Qualcomm

Senior AI Performance Architect

Raleigh, NC · On-site

$162K/yr

The AI Architecture team is comprised of experts that span the full gamut from software ... years Hardware Engineering experience defining architecture of GPUs or accelerators used for ...

Qualcomm

Machine Learning Compiler

Raleigh, NC · On-site

... the AI software stack. • Ensure robust testing, profiling, and performance tuning across diverse ML models and hardware targets. • Manage and mentor a team of engineers, fostering technical ...

Qualcomm

Machine Learning Compiler

Raleigh, NC · On-site

$160K - $240K/yr

... AI software stack. * Ensure robust testing, profiling, and performance tuning across diverse ML models and hardware targets. Team & Project Management: * Manage and mentor a team of engineers ...

Nvidia

Senior Developer Technology Engineer - AI

Durham, NC · Hybrid

$52.75 - $69.50/hr

Influence the design of next-generation hardware architectures, software, and programming models in ... Come, join our AI Compute DevTech team and help build the real-time, cost-effective computing ...

Nvidia

Senior Applied AI Engineer

Durham, NC

$101K - $138K/yr

Collaborate across hardware, software, and research teams, mentoring and supporting peers while ... MS or higher degree (or equivalent experience) in Computer Science, Engineering, AI, or a related ...

next page

Showing results 1-20

All JobsAi Hardware Engineer JobsAi Hardware Engineer Jobs in Raleigh, NC

Ai Hardware Engineer information

See Raleigh, NC salary details

$49.6K

$142.1K

$191K

How much do ai hardware engineer jobs pay per year?

As of Aug 8, 2026, the average yearly pay for ai hardware engineer in Raleigh, NC is $142,148.00, according to ZipRecruiter salary data. Most workers in this role earn between $120,100.00 and $158,400.00 per year, depending on experience, location, and employer.

What are the key skills and qualifications needed to thrive as an AI hardware engineer, and why are they important?

To thrive as an AI Hardware Engineer, you need strong foundations in computer engineering, digital logic design, and knowledge of machine learning algorithms, typically supported by a degree in electrical engineering or computer engineering. Familiarity with hardware description languages (such as Verilog or VHDL), FPGA/ASIC development tools, and experience with simulation and debugging systems are essential. Problem-solving abilities, attention to detail, and effective teamwork are standout soft skills for this role. These skills are crucial for creating efficient, reliable AI hardware solutions and collaborating across multidisciplinary teams to advance cutting-edge technologies.

What is an AI hardware engineer?

AI Hardware Engineers are professionals who design, develop, and optimize computer hardware systems specifically for artificial intelligence applications. Their work involves creating specialized processors, such as GPUs, TPUs, or custom chips, that accelerate machine learning tasks and handle large-scale data processing efficiently. They collaborate with software engineers to ensure hardware and AI models work seamlessly together, improving performance, power efficiency, and scalability of AI systems. AI Hardware Engineers play a crucial role in advancing technologies used in data centers, autonomous vehicles, robotics, and edge devices.

What is the difference between Ai Hardware Engineer vs AI Software Engineer?

AspectAi Hardware EngineerAI Software Engineer
Required CredentialsBachelor's in Electrical Engineering, Computer Engineering, or related fields; knowledge of hardware design and programmingBachelor's or higher in Computer Science, Software Engineering, or related; proficiency in programming languages and AI frameworks
Work EnvironmentDesigning, testing, and developing AI hardware components in labs or manufacturing settingsDeveloping AI algorithms and applications in software development environments
Employer & Industry UsageTech companies, hardware manufacturers, research labsSoftware firms, tech startups, AI research organizations

While both roles focus on AI, Ai Hardware Engineers specialize in creating the physical components that enable AI systems, whereas AI Software Engineers develop the algorithms and software that run on hardware platforms. Understanding these differences helps job seekers target the right roles based on their skills and interests.

How do AI hardware engineers typically collaborate with software and data teams during the development process?

AI Hardware Engineers work closely with software developers and data scientists to ensure that hardware architectures are optimized for specific AI workloads. Collaboration often involves regular meetings to discuss performance requirements, compatibility challenges, and customizations needed for machine learning models. By maintaining open communication channels, hardware engineers can adapt designs based on feedback from software and data teams, ensuring seamless integration and optimal system performance.
What are popular job titles related to Ai Hardware Engineer jobs in Raleigh, NC? For Ai Hardware Engineer jobs in Raleigh, NC, the most frequently searched job titles are:
What job categories do people searching Ai Hardware Engineer jobs in Raleigh, NC look for? The top searched job categories for Ai Hardware Engineer jobs in Raleigh, NC are:
What cities near Raleigh, NC are hiring for Ai Hardware Engineer jobs? Cities near Raleigh, NC with the most Ai Hardware Engineer job openings:
Ai Hardware Engineer jobs near you
Infographic showing various Ai Hardware Engineer job openings in Raleigh, NC as of August 2026, with employment types broken down into 74% Full Time, 20% Part Time, and 6% Contract. Highlights an 67% Physical, 3% Hybrid, and 30% Remote job distribution, with an average salary of $142,148 per year, or $68.3 per hour.

Senior Software Engineer - GPU Local AI Platforms

Nvidia

Durham, NC

$118K - $156K/yr

Full-time

Posted 15 days ago

Nvidia rating

9.6

Company rating: 9.6 out of 10

Based on 17 frontline employees who took The Breakroom Quiz

8th of 242 rated software companies

Job description

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It's a unique legacy of innovation that's fueled by great technology-and amazing people. Today, we're tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what's never been done before takes vision, innovation, and the world's best talent. As an NVIDIAN, you'll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.

NVIDIA's Local AI team is building the software stack that makes large language models and generative AI applications run at maximum efficiency on NVIDIA edge AI hardware. The AI ecosystem moves fast; our job is to make sure end users get the best experience. We own the platform - performance, CI/CD pipelines, validated recipes, and model bring-up infrastructure - that lets developers run groundbreaking LLMs out of the box. The open-source community builds fast; our platform is what turns community innovation into something developers and partners can rely on at scale.

What you'll be doing:

  • Track and evaluate innovations in leading open-source LLM inference frameworks - identify performance-critical features and algorithmic improvements relevant to NVIDIA edge AI hardware

  • Analyze how new model architectures and inference algorithms (attention variants, MoE routing, speculative decoding, multi-token prediction, quantized inference) map onto NVIDIA GPU architecture - identify mismatch, fallback paths, and optimization opportunities

  • Characterize multi-node inference behavior: collective communication primitives (NCCL/RCCL), topology-aware all-reduce strategies, and parallelism efficiency on edge cluster configurations

  • Produce performance analysis reports mapping theoretical hardware limits (memory bandwidth, FLOP/s, interconnect throughput) to observed inference throughput, latency, and utilization

  • Own the model validation workflow for new model releases: architecture compatibility assessment, inference recipe development, performance characterization, and publication to developer recipe sites

  • Develop and maintain developer-facing inference recipes: keep them accurate as frameworks evolve, automate staleness detection, and build feedback loops from CI results to recipe updates

  • Engage with community and partners on model bring-up questions; serve as the technical point of contact for hardware-specific inference issues related to partner concerns

What we need to see:

  • BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or equivalent experience.

  • 12+ years of software engineering with depth in GPU computing, ML systems, or high-performance inference

  • Strong Python or C++ programming, software design, and software engineering skills.

  • Hands-on experience with GPU kernel development or optimization (CUDA/C++, Triton, or equivalent) - you understand how thread blocks, memory hierarchy, and warp execution affect real-world performance

  • Working knowledge of LLM inference internals: attention mechanisms, KV-cache management, continuous batching, quantization formats, and tensor parallelism

  • Container engineering expertise: multi-architecture Docker or OCI builds, layer optimization, runtime configuration, NVIDIA Container Toolkit

  • Strong analytical skills: ability to form a performance hypothesis, design an experiment, interpret results, and communicate findings clearly

Widely considered to be one of the technology world's most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package. As you plan your future, see what we can offer to you and your family www.nvidiabenefits.com/

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 224,000 USD - 356,500 USD for Level 5, and 272,000 USD - 431,250 USD for Level 6.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until July 28, 2026.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

What Nvidia employees say

Pay

Benefits

Hours and flexibility

Workplace

Get the full story on Breakroom

Nvidia logo

About Nvidia

Sourced by ZipRecruiter

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It's a unique legacy of innovation that's fueled by great technology--and amazing people. Today, we're tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what's never been done before takes vision, innovation, and the world's best talent.

Industry

Computer and electronic product manufacturing

Company size

10,000+ Employees

Headquarters location

Santa Clara, CA, US

Year founded

1993

View All Nvidia Jobs

Apply