Job Summary:
NVIDIA has been transforming computer graphics and accelerated computing for over 25 years. They are seeking a Senior Site Reliability Engineer to join their Compute Farm team, responsible for ensuring the reliability and performance of critical systems while leveraging AI to deliver innovative solutions.
Responsibilities:
• Own SRE solutions end‑to‑end, from design and implementation to operation and continuous improvement, ensuring they integrate cleanly with HPC schedulers, storage, and network fabrics.
• Use IaC(Infrastructure‑as‑Code) and config management to standardize and automate provisioning everywhere.
• Deliver solutions in a globally distributed, multi‑cloud hybrid environment – On‑prem, AWS, GCP, and OCI.
• Design for failure with redundancy, failure domains, progressive delivery, and strict change control.
• Ensure the highest level of uptime and Quality of Service (QoS) for internal customers through operational excellence.
• Conduct capacity management and planning to meet ongoing operational needs.
• Detects performance issues and recommends solutions to maintain world‑class service quality.
• Collaborate with various teams in a fast‑paced environment to ensure seamless project completion.
• Participate in on-call, incident reviews, assist in root cause identification, and produce high-quality RCA reports.
Qualifications:
Required:
• B.S. degree in Computer Science or related technical field (or equivalent experience) with 5+ years professional experience building and supporting critical services.
• Experience supporting large‑scale HPC clusters using Slurm, LSF or Kubernetes clusters, including setup, tuning, and troubleshooting.
• Proficiency in modern CI/CD techniques, and Infrastructure as Code (IaC) for managing services.
• Strong experience crafting large-scale infrastructure platforms for automated host lifecycle management, fleet reliability/auto-healing, E2E observability or data-driven operations (AIOps/ML-driven signals) that materially reduce manual intervention.
• Proficient in monitoring, metrics, container management, and log collection tools.
• 5+ years of coding/scripting experience in at least two high‑level programming languages such as Python, Go, Perl, or Ruby.
• Mentored other engineers and influenced technical direction through design reviews, architecture documents, and strong partnership with product and leadership.
• Creative problem solver with excellent debugging skills and strong communication and documentation abilities.
Preferred:
• Published technical write‑ups or talks (conference presentations, meetups, engineering blogs) that deep‑dive into real‑world reliability, observability, or large‑scale HPC/SRE problems and their solutions.
• Maintainer or co‑maintainer responsibilities for an open source component used in production (plugins, operators, exporters, controllers, or SDKs) at large scale.
Company:
NVIDIA is a computing platform company operating at the intersection of graphics, HPC, and AI. Founded in 1993, the company is headquartered in Santa Clara, USA, with a team of 10001+ employees. The company is currently Late Stage.