Redmond, WA · On-site
$137K - $180K/yr
As a member of our AI Software Resiliency team, you will play a pivotal role in defining and ... Hands-on experience in training models or working with model training teams. * Hands-on experience ...
Redmond, WA · On-site
$137K - $180K/yr
As a member of our AI Software Resiliency team, you will play a pivotal role in defining and ... Hands-on experience in training models or working with model training teams. * Hands-on experience ...
Redmond, WA · On-site
$116K - $159K/yr
Define and develop networking requirements for large-scale AI training and inference clusters. * Collaborate with silicon, system software, firmware, hardware, and Azure infrastructure teams to ...
Redmond, WA · On-site
$116K - $159K/yr
Define and develop networking requirements for large-scale AI training and inference clusters. * Collaborate with silicon, system software, firmware, hardware, and Azure infrastructure teams to ...
The role maintains the knowledge base that documents how AI is being used firm-wide and keeps training content and resources current to support both. As a member of the Data & AI team, this role ...
The role maintains the knowledge base that documents how AI is being used firm-wide and keeps training content and resources current to support both. As a member of the Data & AI team, this role ...
The role maintains the knowledge base that documents how AI is being used firm-wide and keeps training content and resources current to support both. As a member of the Data & AI team, this role ...
The role maintains the knowledge base that documents how AI is being used firm-wide and keeps training content and resources current to support both. As a member of the Data & AI team, this role ...
The role maintains the knowledge base that documents how AI is being used firm-wide and keeps training content and resources current to support both. As a member of the Data & AI team, this role ...
The role maintains the knowledge base that documents how AI is being used firm-wide and keeps training content and resources current to support both. As a member of the Data & AI team, this role ...
The role maintains the knowledge base that documents how AI is being used firm-wide and keeps training content and resources current to support both. As a member of the Data & AI team, this role ...
The role maintains the knowledge base that documents how AI is being used firm-wide and keeps training content and resources current to support both. As a member of the Data & AI team, this role ...
Bellevue, WA · Remote
$50 - $100/hr
To succeed in this position, you should have expert-level financial reasoning and formal training ... Give AI chatbots diverse and complex problems and evaluate their outputs * Evaluate the quality ...
Quick apply
Bellevue, WA · Remote
$50 - $100/hr
To succeed in this position, you should have expert-level financial reasoning and formal training ... Give AI chatbots diverse and complex problems and evaluate their outputs * Evaluate the quality ...
Seattle, WA · Remote
$50 - $100/hr
To succeed in this position, you should have expert-level financial reasoning and formal training ... Give AI chatbots diverse and complex problems and evaluate their outputs * Evaluate the quality ...
Quick apply
Seattle, WA · Remote
$50 - $100/hr
To succeed in this position, you should have expert-level financial reasoning and formal training ... Give AI chatbots diverse and complex problems and evaluate their outputs * Evaluate the quality ...
Renton, WA · Remote
$50 - $100/hr
To succeed in this position, you should have expert-level financial reasoning and formal training ... Give AI chatbots diverse and complex problems and evaluate their outputs * Evaluate the quality ...
Quick apply
Renton, WA · Remote
$50 - $100/hr
To succeed in this position, you should have expert-level financial reasoning and formal training ... Give AI chatbots diverse and complex problems and evaluate their outputs * Evaluate the quality ...
Everett, WA · Remote
$50 - $100/hr
To succeed in this position, you should have expert-level financial reasoning and formal training ... Give AI chatbots diverse and complex problems and evaluate their outputs * Evaluate the quality ...
Quick apply
Everett, WA · Remote
$50 - $100/hr
To succeed in this position, you should have expert-level financial reasoning and formal training ... Give AI chatbots diverse and complex problems and evaluate their outputs * Evaluate the quality ...
Bellevue, WA · On-site
$145K - $188K/yr
Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior ... Applied Training is the layer on top of that infrastructure that helps researchers and customers ...
Quick apply
Bellevue, WA · On-site
$145K - $188K/yr
Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior ... Applied Training is the layer on top of that infrastructure that helps researchers and customers ...
Design and implement scalable distributed systems supporting AI training and inference environments. * Collaborate with networking, AI infrastructure, hardware, and cloud platform teams to deliver ...
Design and implement scalable distributed systems supporting AI training and inference environments. * Collaborate with networking, AI infrastructure, hardware, and cloud platform teams to deliver ...
Everett, WA · Remote
$140 - $350/hr
No prior experience in AI is required -- your domain knowledge is what matters. This opportunity is tailored for US-based legal professionals with backgrounds at top-tier law firms, seeking to ...
Everett, WA · Remote
$140 - $350/hr
No prior experience in AI is required -- your domain knowledge is what matters. This opportunity is tailored for US-based legal professionals with backgrounds at top-tier law firms, seeking to ...
Seattle, WA · Remote
$140 - $350/hr
No prior experience in AI is required -- your domain knowledge is what matters. This opportunity is tailored for US-based legal professionals with backgrounds at top-tier law firms, seeking to ...
Seattle, WA · Remote
$140 - $350/hr
No prior experience in AI is required -- your domain knowledge is what matters. This opportunity is tailored for US-based legal professionals with backgrounds at top-tier law firms, seeking to ...
Renton, WA · Remote
$140 - $350/hr
No prior experience in AI is required -- your domain knowledge is what matters. This opportunity is tailored for US-based legal professionals with backgrounds at top-tier law firms, seeking to ...
Renton, WA · Remote
$140 - $350/hr
No prior experience in AI is required -- your domain knowledge is what matters. This opportunity is tailored for US-based legal professionals with backgrounds at top-tier law firms, seeking to ...
Bellevue, WA · Remote
$140 - $350/hr
No prior experience in AI is required -- your domain knowledge is what matters. This opportunity is tailored for US-based legal professionals with backgrounds at top-tier law firms, seeking to ...
Bellevue, WA · Remote
$140 - $350/hr
No prior experience in AI is required -- your domain knowledge is what matters. This opportunity is tailored for US-based legal professionals with backgrounds at top-tier law firms, seeking to ...
Redmond, WA · On-site
$121K - $165K/yr
This team focuses on developing tools for optimizing efficiency and resiliency of AI workloads - pre-training, post-training, inference. Our objective is to deliver a stable, scalable environment for ...
Redmond, WA · On-site
$121K - $165K/yr
This team focuses on developing tools for optimizing efficiency and resiliency of AI workloads - pre-training, post-training, inference. Our objective is to deliver a stable, scalable environment for ...
Bellevue, WA · On-site
$70 - $75/hr
Build and optimize data pipelines using Azure Data Factory, Databricks, Snowflake, or PySpark to support AI model training and inference. * Apply prompt engineering and fine-tuning techniques to ...
Quick apply
Bellevue, WA · On-site
$70 - $75/hr
Build and optimize data pipelines using Azure Data Factory, Databricks, Snowflake, or PySpark to support AI model training and inference. * Apply prompt engineering and fine-tuning techniques to ...
Everett, WA · Remote
$90 - $130/hr
Prior exposure to AI, legal tech, or training initiatives. * Experience working with private equity firms. Why Join: * This is an opportunity to work at the intersection of law and technology. * You ...
Everett, WA · Remote
$90 - $130/hr
Prior exposure to AI, legal tech, or training initiatives. * Experience working with private equity firms. Why Join: * This is an opportunity to work at the intersection of law and technology. * You ...
Bellevue, WA · Remote
$90 - $130/hr
Prior exposure to AI, legal tech, or training initiatives. * Experience working with private equity firms. Why Join: * This is an opportunity to work at the intersection of law and technology. * You ...
Bellevue, WA · Remote
$90 - $130/hr
Prior exposure to AI, legal tech, or training initiatives. * Experience working with private equity firms. Why Join: * This is an opportunity to work at the intersection of law and technology. * You ...
$35.8K - $43.9K
15% of jobs
$43.9K - $52K
7% of jobs
$54.5K is the 25th percentile. Wages below this are outliers.
$52K - $60.2K
10% of jobs
$60.2K - $68.3K
11% of jobs
The median wage is $70.6K / yr.
$68.3K - $76.4K
22% of jobs
$76.4K - $84.6K
4% of jobs
$90.4K is the 75th percentile. Wages above this are outliers.
$84.6K - $92.7K
7% of jobs
$92.7K - $100.8K
7% of jobs
$100.8K - $108.9K
4% of jobs
$108.9K - $117.1K
5% of jobs
$117.1K - $125.2K
6% of jobs
$35.8K
$76.8K
$125.2K
An AI Training job involves preparing and refining data to help improve artificial intelligence models. This can include labeling data, reviewing AI-generated outputs, and providing feedback to enhance accuracy. AI Trainers work with machine learning teams to fine-tune models for tasks like chatbots, image recognition, or language processing. Strong attention to detail and an understanding of AI behavior are key skills for this role.
AI Training specialists are primarily responsible for curating, labeling, and managing large datasets that are used to train machine learning models. They often collaborate closely with data scientists and engineers to ensure data quality, clarity of labeling guidelines, and that project requirements are met. Daily tasks can include annotating images or text, performing data validation, refining annotation protocols, and reviewing outputs generated by AI models. This role frequently involves teamwork as well as independent problem-solving, offering opportunities to learn about AI development workflows and expand technical knowledge.
To thrive in AI Training, you need a solid understanding of machine learning concepts, data annotation, and statistical analysis, often supported by a background in computer science or related fields. Familiarity with tools such as TensorFlow, PyTorch, labeling software, and experience with data management systems is highly valued. Strong attention to detail, effective communication, and critical thinking are important soft skills for this role. These skills ensure the accurate preparation and handling of training data, which directly impacts the quality and success of AI models.
$137K - $180K/yr
Full-time
Re-posted 19 days ago
9.6
Based on 17 frontline employees who took The Breakroom Quiz
8th of 242 rated software companies
We are now looking for a Senior Software Engineer for AI Resiliency!
At NVIDIA, we are pushing the boundaries of what's possible in AI. We are currently seeking a Senior Software Engineer to lead the development of AI software resiliency for the most powerful AI supercomputers in the world. As a member of our AI Software Resiliency team, you will play a pivotal role in defining and implementing critical resiliency features for AI supercomputers at a scale of 100,000+ GPUs. Your expertise will be crucial in driving down cluster downtime towards zero, ensuring that our AI systems remain robust and reliable at all times.
What You'll Be Doing:
Develop AI Software Resiliency Features: Implement and optimize software features that improve AI system reliability at a massive scale, such as fast checkpoint-recovery, error detection, error isolation, and straggler/hang detection.
Hands-On Coding & Optimization: Contribute to large-scale distributed systems with high-quality, production-level C++ and Python code. Enhance performance for AI workloads running on thousands of GPUs.
Fault Tolerance & Debugging: Work on AI system error handling, implementing techniques to detect silent data corruption (SDC) and other failure scenarios. Assist in developing monitoring tools for proactive failure mitigation.
Collaborate Across Teams: Work closely with senior engineers, AI researchers, and hardware/software teams to integrate resiliency features into AI frameworks like PyTorch and JAX/XLA.
Testing & Automation: Develop and implement tests to ensure robustness, scalability, and efficiency of resiliency mechanisms. Contribute to CI/CD pipelines to automate validation of AI workloads.
Support Production Deployments: Assist in debugging and performance tuning large-scale AI workloads in cloud and HPC environments, ensuring seamless operation of AI training and inference workloads.
What We Need to See:
You've achieved a Bachelor's, Master's or PhD in Computer Science, Electrical Engineering, or a related field, or equivalent experience.
Proficiency in C++ and Python, with experience in writing efficient, high-performance code.
6+ years of relevant experience
Strong understanding of distributed systems concepts, parallel programming, and fault tolerance in large-scale computing environments.
Familiarity with AI frameworks such as PyTorch, JAX/XLA, TensorFlow, or similar.
Experience with debugging and profiling tools (e.g., gdb, perf, valgrind, NVIDIA Nsight).
Excellent problem-solving skills and ability to work in a fast-paced, highly collaborative environment.
Ways to Stand Out From the Crowd:
Hands-on experience in training models or working with model training teams.
Hands-on experience with CUDA, NCCL, or MPI for GPU-accelerated computing, especially at extreme-scale.
Knowledge of checkpointing strategies, error mitigation, or fault-tolerant computing in AI training.
Experience working with large-scale AI clusters, HPC environments, or cloud-based AI workloads.
Strong systems programming skills and experience with low-level performance tuning.
As part of the AI Resiliency team at NVIDIA, you'll work alongside world-class engineers solving some of the hardest challenges in AI infrastructure. You'll have the opportunity to contribute directly to making AI training and inference more reliable, scalable, and efficient. If you're passionate about AI, distributed systems, and high-performance computing, we want to hear from you!
Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD.You will also be eligible for equity and benefits.
This posting is for an existing vacancy.
NVIDIA uses AI tools in its recruiting processes.
NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
Sourced by ZipRecruiter
NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It's a unique legacy of innovation that's fueled by great technology--and amazing people. Today, we're tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what's never been done before takes vision, innovation, and the world's best talent.
Computer and electronic product manufacturing
10,000+ Employees
Santa Clara, CA, US
1993
