The Deep Learning Inference team develops and optimizes open-source frameworks that make AI ... Our work enables developers worldwide to harness NVIDIA accelerators for real-time inference at ...
The Deep Learning Inference team develops and optimizes open-source frameworks that make AI ... Our work enables developers worldwide to harness NVIDIA accelerators for real-time inference at ...
Senior Deep Learning Software Engineer, Inference
New York, NY · On-site +1
$134K - $176K/yr
Scale performance of DL models across different architectures and types of NVIDIA accelerators ... Contribute to deep learning software projects, such as PyTorch, vLLM, and SGLang to drive ...
Senior Deep Learning Software Engineer, Inference
New York, NY · On-site +1
$134K - $176K/yr
Scale performance of DL models across different architectures and types of NVIDIA accelerators ... Contribute to deep learning software projects, such as PyTorch, vLLM, and SGLang to drive ...
Senior Software Engineer (Architecture)
New York, NY · On-site
$134K - $176K/yr
Research and design new software and hardware AI solutions, involving simulators, optimizing compilers, code generators, and runtime execution frameworks for deep learning accelerators. * Evaluate ...
Quick apply
Senior Software Engineer (Architecture)
New York, NY · On-site
$134K - $176K/yr
Research and design new software and hardware AI solutions, involving simulators, optimizing compilers, code generators, and runtime execution frameworks for deep learning accelerators. * Evaluate ...
... deep learning hardware components optimized for AR/VR systems. You will be part of our efforts to ... Implement and/or enhance code generation targeting machine learning accelerators * Work with ...
... deep learning hardware components optimized for AR/VR systems. You will be part of our efforts to ... Implement and/or enhance code generation targeting machine learning accelerators * Work with ...
Deep hands-on experience with a modern deep-learning framework, especially PyTorch (JAX or ... Familiarity with distributed training, inference optimization, or GPU/accelerator performance work
Quick apply
Deep hands-on experience with a modern deep-learning framework, especially PyTorch (JAX or ... Familiarity with distributed training, inference optimization, or GPU/accelerator performance work
Deep hands-on experience with a modern deep-learning framework, especially PyTorch (JAX or ... Familiarity with distributed training, inference optimization, or GPU/accelerator performance work
Deep hands-on experience with a modern deep-learning framework, especially PyTorch (JAX or ... Familiarity with distributed training, inference optimization, or GPU/accelerator performance work
Deep hands-on experience with a modern deep-learning framework, especially PyTorch (JAX or ... Familiarity with distributed training, inference optimization, or GPU/accelerator performance work
Deep hands-on experience with a modern deep-learning framework, especially PyTorch (JAX or ... Familiarity with distributed training, inference optimization, or GPU/accelerator performance work
Drive business development through proposals, executive client pitches, solution accelerators ... Deep expertise in modern GenAI patterns, including advanced RAG, embeddings, vector and hybrid ...
Drive business development through proposals, executive client pitches, solution accelerators ... Deep expertise in modern GenAI patterns, including advanced RAG, embeddings, vector and hybrid ...
AMI Scientist
New York, NY · On-site
Understanding of machine learning fundamentals, large-scale training, and accelerator-based (GPU or TPU) compute environments Preferred Qualifications: * Deep expertise in at least one of the ...
AMI Scientist
New York, NY · On-site
Understanding of machine learning fundamentals, large-scale training, and accelerator-based (GPU or TPU) compute environments Preferred Qualifications: * Deep expertise in at least one of the ...
Drive business development through proposals, executive client pitches, solution accelerators ... Deep expertise in modern GenAI patterns, including advanced RAG, embeddings, vector and hybrid ...
Drive business development through proposals, executive client pitches, solution accelerators ... Deep expertise in modern GenAI patterns, including advanced RAG, embeddings, vector and hybrid ...
Drive business development through proposals, executive client pitches, solution accelerators ... Deep expertise in modern GenAI patterns, including advanced RAG, embeddings, vector and hybrid ...
Drive business development through proposals, executive client pitches, solution accelerators ... Deep expertise in modern GenAI patterns, including advanced RAG, embeddings, vector and hybrid ...
... learning accelerators. FPGAs and ASICs are critical pieces of our technology stack. We are looking ... Deep knowledge of FPGA internals and/or ASIC * Excellent digital logic design, optimization ...
... learning accelerators. FPGAs and ASICs are critical pieces of our technology stack. We are looking ... Deep knowledge of FPGA internals and/or ASIC * Excellent digital logic design, optimization ...
Understanding of machine learning fundamentals, large-scale training, and accelerator-based (GPU or TPU) compute environments Preferred Qualifications: * Deep expertise in at least one of the ...
Understanding of machine learning fundamentals, large-scale training, and accelerator-based (GPU or TPU) compute environments Preferred Qualifications: * Deep expertise in at least one of the ...
AMI Engineer
New York, NY · On-site
Understanding of machine learning fundamentals, large-scale training, and accelerator-based (GPU or ... Proficiency in a deep learning framework (PyTorch or JAX), especially for distributed training and ...
AMI Engineer
New York, NY · On-site
Understanding of machine learning fundamentals, large-scale training, and accelerator-based (GPU or ... Proficiency in a deep learning framework (PyTorch or JAX), especially for distributed training and ...
Senior AI Architect
Manhattan, NY · On-site
Contribute to building internal IP, accelerators, and reference architectures that leverage ... Solid understanding of the data and AI landscape: classical ML, deep learning, generative AI ...
Senior AI Architect
Manhattan, NY · On-site
Contribute to building internal IP, accelerators, and reference architectures that leverage ... Solid understanding of the data and AI landscape: classical ML, deep learning, generative AI ...
Senior AI Architect
Manhattan, NY · On-site
Contribute to building internal IP, accelerators, and reference architectures that leverage ... Solid understanding of the data and AI landscape: classical ML, deep learning, generative AI ...
Senior AI Architect
Manhattan, NY · On-site
Contribute to building internal IP, accelerators, and reference architectures that leverage ... Solid understanding of the data and AI landscape: classical ML, deep learning, generative AI ...
Senior AI Architect
Manhattan, NY · On-site
Contribute to building internal IP, accelerators, and reference architectures that leverage ... Solid understanding of the data and AI landscape: classical ML, deep learning, generative AI ...
Senior AI Architect
Manhattan, NY · On-site
Contribute to building internal IP, accelerators, and reference architectures that leverage ... Solid understanding of the data and AI landscape: classical ML, deep learning, generative AI ...
Senior AI Architect
Manhattan, NY · On-site
Contribute to building internal IP, accelerators, and reference architectures that leverage ... Solid understanding of the data and AI landscape: classical ML, deep learning, generative AI ...
Quick apply
Senior AI Architect
Manhattan, NY · On-site
Contribute to building internal IP, accelerators, and reference architectures that leverage ... Solid understanding of the data and AI landscape: classical ML, deep learning, generative AI ...
Senior AI Architect
Manhattan, NY · On-site
Contribute to building internal IP, accelerators, and reference architectures that leverage ... Solid understanding of the data and AI landscape: classical ML, deep learning, generative AI ...
Senior AI Architect
Manhattan, NY · On-site
Contribute to building internal IP, accelerators, and reference architectures that leverage ... Solid understanding of the data and AI landscape: classical ML, deep learning, generative AI ...
Research Scientist, AI & Systems Co-Design (PhD)
New York, NY · On-site
$122K/yr
The team invests significantly into model optimization on existing accelerator systems and guiding ... Experience or knowledge of training/inference of large-scale deep learning models * Familiarity ...
Research Scientist, AI & Systems Co-Design (PhD)
New York, NY · On-site
$122K/yr
The team invests significantly into model optimization on existing accelerator systems and guiding ... Experience or knowledge of training/inference of large-scale deep learning models * Familiarity ...
Deep Learning Accelerator information
What is a deep learning accelerator?
Deep Learning Accelerators are specialized hardware or systems designed to speed up the processing and training of deep learning algorithms, such as neural networks. They are optimized for the heavy computational demands of tasks like image recognition, natural language processing, and other AI applications. Examples include Graphics Processing Units (GPUs), Tensor Processing Units (TPUs), and custom-designed chips like Application-Specific Integrated Circuits (ASICs) and Field-Programmable Gate Arrays (FPGAs). These accelerators enable faster data processing, lower power consumption, and improved efficiency compared to general-purpose CPUs. As AI applications grow, the use of deep learning accelerators is becoming increasingly important in both research and industry.
What skills and qualifications are needed to thrive as a deep learning accelerator engineer?
To thrive as a Deep Learning Accelerator Engineer, you need a strong background in computer engineering, digital design, and machine learning, typically supported by a degree in computer science or electrical engineering. Experience with hardware description languages (such as Verilog or VHDL), FPGA/ASIC toolchains, and familiarity with deep learning frameworks like TensorFlow or PyTorch is essential. Problem-solving, teamwork, and effective communication are crucial soft skills for collaborating with cross-functional teams and translating algorithmic requirements into efficient hardware solutions. These skills are vital to designing high-performance, energy-efficient hardware accelerators that advance AI capabilities and meet industry demands.
What are the main challenges faced when optimizing deep learning models for hardware accelerators?
One of the primary challenges in this role is bridging the gap between deep learning model requirements and the constraints of specialized hardware, such as GPUs, TPUs, or custom ASICs. This often involves model quantization, memory optimization, and adapting algorithms to exploit hardware parallelism while maintaining accuracy and efficiency. Collaboration with both hardware engineers and software developers is essential to ensure models run efficiently on target platforms, and staying current with evolving accelerator architectures is key to long-term success.
What is the difference between Deep Learning Accelerator vs Machine Learning Engineer?
| Aspect | Deep Learning Accelerator | Machine Learning Engineer |
|---|---|---|
| Required Credentials | Knowledge of hardware design, FPGA/ASIC programming, deep learning frameworks | Degree in Computer Science, Data Science, or related fields; experience with ML frameworks |
| Work Environment | Hardware development labs, embedded systems, AI hardware companies | Software development environments, tech companies, research labs |
| Industry Usage | AI hardware manufacturing, embedded AI solutions | AI/ML software development, data analysis, model deployment |
| Search & Comparison Intent | Focus on hardware acceleration, AI hardware design | Focus on software development, model building |
Deep Learning Accelerators specialize in hardware design and optimization for AI workloads, working closely with hardware and embedded systems. Machine Learning Engineers develop and deploy ML models primarily through software, focusing on algorithms and data. While both roles involve AI, their core skills, work environments, and industry applications differ significantly.
What cities in New York are hiring for Deep Learning Accelerator jobs?
Cities in New York with the most Deep Learning Accelerator job openings:
Engineering Manager, Deep Learning Inference
New York, NY • On-site, Remote
Nvidia
• Computer and Electronic Product Manufacturing • 10K+ employees10,000+ employees
Full-time
Re-posted 5 days ago
Key responsibilities
Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
Drive the strategy, roadmap, and execution of NVIDIA's inference frameworks engineering, focusing on Client AI.
Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications.
Nvidia rating
9.6
Based on 18 frontline employees who took The Breakroom Quiz
Job description
NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing the state of AI model deployment. You will shape the software powering today's most sophisticated AI systems - from large language models to multimodal generative AI - all accelerated on NVIDIA GPUs. The Deep Learning Inference team develops and optimizes open-source frameworks that make AI deployment scalable, efficient, and accessible - including vLLM / SGLang, and FlashInfer. Our work enables developers worldwide to harness NVIDIA accelerators for real-time inference at every scale, from datacenter clusters to edge devices.
What you'll be doing:
Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
Drive the strategy, roadmap, and execution of NVIDIA's inference frameworks engineering, focusing on Client AI.
Partner with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators.
Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications.
Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications (NIXL, NCCL, NVSHMEM).
Represent the team in roadmap and planning discussions, ensuring alignment with NVIDIA's broader AI and software strategies.
Foster a culture of technical excellence, open collaboration, and continuous innovation.
What we need to see:
MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a related field.
6+ overall years of software development experience, including 3+ years in technical leadership or engineering management.
Strong background in C/C++ software design and development; proficiency in Python is a plus.
Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization.
Proven record of deploying or optimizing deep learning models in production environments.
Experience leading teams using Agile or collaborative software development practices.
Ways to Stand out from The Crowd:
Significant open-source contributions to deep learning or inference frameworks such as PyTorch, vLLM / SGLang, Triton, or TensorRT-LLM.
Deep understanding of multi-GPU communications (NIXL, NCCL, NVSHMEM) and distributed inference architectures.
Expertise in performance modeling, profiling, and system-level optimization across CPU and GPU platforms.
Proven ability to mentor engineers, guide architectural decisions, and deliver complex projects with measurable impact.
Publications, patents, or talks on LLM serving, model optimization, or GPU performance engineering.
With highly competitive salaries and a comprehensive benefits package, NVIDIA is widely considered to be one of the technology world's most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us and, our rapid growth means endless opportunities for career advancement. If you're a passionate technical leader ready to shape the future of AI inference frameworks - and build the software that powers the world's most advanced models - we'd love to hear from you.
#LI-Hybrid
Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD for Level 2, and 224,000 USD - 356,500 USD for Level 3.You will also be eligible for equity and benefits.
This posting is for an existing vacancy.
NVIDIA uses AI tools in its recruiting processes.
NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
About Nvidia
Sourced by ZipRecruiter
NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It's a unique legacy of innovation that's fueled by great technology--and amazing people. Today, we're tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what's never been done before takes vision, innovation, and the world's best talent.
Industry
Computer and electronic product manufacturing
Company size
10,000+ Employees
Headquarters location
Santa Clara, CA, US