Senior Software Dev Engineer, ECNitro

Overview

Job  | Amazon Development Center U.S., Inc. Join the EC2 Nitro Machine Learning Systems team to revolutionize supercomputing in the cloud. We are seeking an experienced Software Development Engineer to build and optimize infrastructure powering the most computationally intensive AI/ML workloads. In this role, you will establish EC2 as the definitive source for best-known-configurations across diverse ML applications while influencing future accelerated platform designs.

You will bring deep expertise in ML systems performance, working across the full stack from low-level hardware optimization to high-level frameworks. This position offers opportunities to translate state-of-the-art ML research into practical platform improvements, build foundational measurement infrastructure, and directly support customers with performance challenges. If you are passionate about solving complex performance optimization problems at massive scale while directly influencing product strategy, this role provides a significant impact opportunity.

• Design and implement scalable performance measurement infrastructure that serves as the foundation for ML benchmarking across AWS, incorporating metrics like tokens/second, latency, and accelerator utilization.
• Lead technical projects establishing EC2 as the definitive source for ML performance best practices across diverse applications including LLMs, multimodal systems, and emerging model architectures.
• Develop and maintain comprehensive regression testing systems that validate performance across major component releases including frameworks, firmware, drivers, and networking infrastructure.
• Collaborate with hardware engineering teams to influence future accelerator platform designs based on performance insights from state-of-the-art research and customer workloads.
• Build customer relationships by investigating complex performance challenges, developing solutions, and publishing actionable best practices through multiple channels.

Your day revolves around translating technical performance data into actionable business insights while solving complex optimization challenges. You might start by analyzing performance bottlenecks in a customer's large language model training workflow, collaborate with framework engineers to implement optimizations, and present findings at platform design reviews that influence future hardware decisions. You will balance immediate customer needs with long-term infrastructure development and help establish processes for this bootstrap team.

The EC2 Nitro Machine Learning Systems team is responsible for development, operations, and maintenance of scale-out machine learning platforms used for training and inference workloads. We build and optimize the infrastructure that powers some of the most computationally intensive AI/ML workloads in the cloud. Our team is passionate about creating reliable, high-performance systems that enable customers to push the boundaries of what's possible with machine learning.

• 5+ years of non-internship professional software development experience
• 5+ years of programming experience in at least one software programming language
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems
• Experience as a mentor, tech lead, or leading an engineering team
• Knowledge of Machine Learning and LLM fundamentals, including transformer architecture, training/inference life cycles, and optimization techniques

• 5+ years of full software development lifecycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
• Bachelor's degree in computer science or equivalent
• Knowledge of ML frameworks including JAX, PyTorch, vLLM, SGLang, Dynamo, Torch
  
  XLA, and TensorRT
• Knowledge of machine learning model architecture and inference

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best…