Prefix Jobs (NOW HIRING)

Job Summary:
AMD is a company focused on building innovative products that enhance computing experiences across various domains including AI and data centers. They are seeking a Principal GenAI Inference Optimization Engineer to improve the performance and efficiency of generative AI inference workloads on AMD GPU platforms, optimizing latency and throughput for large-scale models.
Responsibilities:
• Optimize performance of GenAI inference workloads on AMD GPU platforms across single-node and distributed environments.
• Improve latency, throughput, and cost efficiency for LLM and multimodal model serving in production.
• Analyze and resolve bottlenecks across compute, memory, and communication (e.g., kernel efficiency, KV-cache usage, memory bandwidth, scheduling).
• Contribute to cross-stack optimizations spanning kernels, runtimes, communication libraries, and inference/serving frameworks (e.g., vLLM, SGLang, Triton, or similar systems).
• Implement and evaluate inference optimization techniques such as batching strategies, quantization, prefix caching, and speculative decoding.
• Support development and optimization of scalable serving systems, including request scheduling and resource utilization.
• Develop and use profiling, benchmarking, and performance analysis tools for inference workloads.
• Collaborate with hardware, compiler, and framework teams to improve overall system performance.
• Contribute to internal tools and, where applicable, open-source projects for inference optimization on AMD platforms.
• Document best practices and contribute to performance guidelines for GenAI deployment.
Qualifications:
Required:
• Strong technical contributor with expertise in GenAI inference optimization, GPU performance, and large-scale serving systems.
• Solid understanding of GPU architecture, memory systems, and communication patterns.
• Ability to improve inference efficiency.
• Comfortable working across multiple layers—from kernels and runtimes to frameworks and serving systems.
• Ability to independently drive optimization efforts while collaborating with cross-functional teams.
• Optimize performance of GenAI inference workloads on AMD GPU platforms across single-node and distributed environments.
• Improve latency, throughput, and cost efficiency for LLM and multimodal model serving in production.
• Analyze and resolve bottlenecks across compute, memory, and communication (e.g., kernel efficiency, KV-cache usage, memory bandwidth, scheduling).
• Contribute to cross-stack optimizations spanning kernels, runtimes, communication libraries, and inference/serving frameworks (e.g., vLLM, SGLang, Triton, or similar systems).
• Implement and evaluate inference optimization techniques such as batching strategies, quantization, prefix caching, and speculative decoding.
• Support development and optimization of scalable serving systems, including request scheduling and resource utilization.
• Develop and use profiling, benchmarking, and performance analysis tools for inference workloads.
• Collaborate with hardware, compiler, and framework teams to improve overall system performance.
• Contribute to internal tools and, where applicable, open-source projects for inference optimization on AMD platforms.
• Document best practices and contribute to performance guidelines for GenAI deployment.
• B.S., M.S. or Ph.D. in Computer Science, Computer Engineering, or a related field preferred, or equivalent industry experience.
Preferred:
• Strong understanding of GPU architecture and performance fundamentals (compute, memory hierarchy, interconnects such as PCIe/Infinity Fabric/RDMA).
• Experience with GenAI inference optimization techniques (e.g., quantization, KV-cache optimization, batching).
• Hands-on experience with inference/serving frameworks such as vLLM, SGLang, Triton, TensorRT-LLM, or similar.
• Experience working on LLM or multimodal inference workloads.
• Familiarity with distributed systems and serving architectures.
• Experience with ML frameworks (PyTorch, JAX, or TensorFlow), especially for inference.
• Proficiency in Python and at least one systems language (C++/CUDA/HIP).
• Experience with profiling, debugging, and performance tuning tools.
• Ability to work collaboratively across teams and deliver impactful optimizations.
Company:
Advanced Micro Devices is a semiconductor company that designs and develops graphics units, processors, and media solutions. Founded in 1969, the company is headquartered in Santa Clara, USA, with a team of 10001+ employees. The company is currently Late Stage.