Senior Software Engineer, AI and DL Kernel Libraries

Overview

We’re looking for outstanding AI systems engineers to develop groundbreaking technologies in the inference systems software stack. As a member of the team, you will build libraries, code generators, and GPU kernel technologies for NVIDIA’s hardware architecture, designing new abstractions, efficient attention kernel implementations, and LLM inference runtimes components to accelerate large language models and other high‑impact AI workloads.

• Innovate and develop new AI systems technologies for efficient inference.
• Design, implement, and optimize kernels for high‑impact AI workloads.
• Design and implement extensible abstractions for LLM serving engines.
• Build efficient just‑in‑time domain‑specific compilers and runtimes.
• Collaborate closely with engineers across deep learning frameworks, libraries, kernels, and GPU architecture teams.
• Contribute to open‑source communities such as Flash Infer, vLLM, and SGLang.

• Master’s degree in Computer Science, Electrical Engineering, or related field (or equivalent experience);
  PhD preferred.
• 6+ years of academic or industry experience in ML/DL systems development.
• Strong experience developing or using deep learning frameworks (e.g., PyTorch, JAX, Tensor Flow, ONNX) and inference engines and runtimes such as vLLM, SGLang, and MLC.
• Excellent Python and C/C++ programming skills.
• Proven experience in GPU kernel development and performance optimizations, especially with CUDA C/C++, cuTile, Triton, or similar.

• Background in domain‑specific compiler and library solutions for LLM inference and training (e.g., Flash Infer, Flash Attention).
• Expertise in inference engines like vLLM and SGLang.
• Expertise in machine learning compilers (e.g., Apache TVM, MLIR).
• Open‑source project ownership or significant contributions.

Base salary is determined by location, experience, and peers, with a range of 184,000 USD to 287,500 USD. You will also be eligible for equity and other benefits.

NVIDIA is committed to fostering an inclusive work environment and is a proud equal‑opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status, or any other characteristic protected by law.