Senior ML Infra Engineer- Distributed Systems

WHAT YOU DO AT AMD CHANGES EVERYTHING

At AMD, our mission is to build great products that accelerate next-generation computing experiences—from AI and data centers, to PCs, gaming and embedded systems. Grounded in a culture of innovation and collaboration, we believe real progress comes from bold ideas, human ingenuity and a shared passion to create something extraordinary. When you join AMD, you’ll discover the real differentiator is our culture.

We push the limits of innovation to solve the world’s most important challenges—striving for execution excellence, while being direct, humble, collaborative, and inclusive of diverse perspectives. Join us as we shape the future of AI and beyond.

Welcome to the Llama team—where curiosity runs wild, and innovation is our natural habitat. We’re a tight-knit group of passionate builders, educators, and AI explorers at AMD, united by a shared mission: to push the boundaries of what’s possible with generative AI and make cutting‑edge knowledge accessible to developers everywhere.

AMD is looking for a Senior Staff AI Infra Engineer who is passionate about improving the performance of key applications and benchmarks, with a special focus on AI/ML workloads and GPU‑accelerated computing. As an Senior Staff Engineer, you will be a technical leader within a core project and will work at the intersection of hardware and software to optimize performance for next‑generation AI applications, including Large Language Models (LLMs) and Agentic AI systems.

You will work with the very latest hardware and software technology, providing technical leadership while driving complex technical initiatives.

The ideal candidate should be passionate about software engineering and possess strong leadership skills to drive sophisticated issues to resolution. Must demonstrate technical depth and breadth in both traditional computing and emerging AI technologies, with the ability to influence technical direction and mentor other engineers. Able to communicate effectively and work optimally with different teams across AMD.

• Lead technical initiatives and provide architectural guidance for AI/ML infrastructure and performance optimization.
• Optimize and accelerate LLM training and inference on AMD GPUs, improving kernel, communication, and end‑to‑end system efficiency.
• Develop and enhance infrastructure supporting LLMs, Agentic AI, and RAG systems.
• Design, build, and optimize AI workloads on GPU clusters, including large‑scale training and inference orchestration, elastic scaling, and workload scheduling across heterogeneous hardware.
• Debug and resolve complex system‑level performance issues across GPU, network, and runtime layers.
• Drive technical excellence, foster cross‑team collaboration, and champion innovation within the organization.

• 5+ years of experience in AI/ML infrastructure, distributed systems, or performance‑critical software development.
• Expert‑level proficiency in C/C++ and Python.
• Solid understanding of transformer‑based architectures and distributed training frameworks such as Megatron‑LM, Deep Speed, and PyTorch Distributed.
• Proven experience optimizing LLM training and inference pipelines, including TP/PP/DP/ZeRO parallelism, quantization, and mixed‑precision techniques.
• Hands‑on experience designing, building, and scaling training or inference platforms using Kubernetes, Ray, or Kubeflow.
• Familiarity with GPU architecture and distributed communication libraries (e.g., NCCL, RCCL, MPI), with the ability to analyze and optimize multi‑GPU training performance.
• Experience with profiling and performance‑analysis tools for GPU optimization and system‑level debugging.
• Demonstrated technical ownership, strong communication, and problem‑solving skills, with a proven record of delivering end‑to‑end AI/ML infrastructure solutions.

• In‑depth experience with the AMD ROCm ecosystem, including HIP kernel optimization for training and inference.
• Hands‑on experience with model optimization techniques such as quantization, pruning, and distillation for efficient deployment.
• Knowledge of GPU architecture, memory…