Senior Software Engineer, Generative AI Systems

Job Overview

NVIDIA is seeking a highly motivated Software Engineer to join our growing AI and Generative AI engineering team. In this role, you will contribute to the design, development, and evaluation of large-scale AI systems powering next‑generation applications in LLMs, agentic AI, retrieval‑augmented generation (RAG), and intelligent automation.

• Design and develop scalable infrastructure for large‑scale ML training, inference, and generative AI systems.
• Build distributed systems and cloud‑native platforms supporting GPU clusters, fault‑tolerant training, and high‑performance AI workloads.
• Develop evaluation frameworks for LLMs and agentic AI systems, including hallucination detection, safety validation, robustness testing, and tool‑calling reliability.
• Architect and optimize RAG pipelines, knowledge management systems, and scalable AI data workflows.
• Build backend services, APIs, and production AI infrastructure using technologies such as FastAPI, Kubernetes, Docker, and modern cloud platforms.
• Develop automated benchmarking, orchestration, and asynchronous processing systems for enterprise AI applications and evaluation platforms.
• Collaborate cross‑functionally with research, product, and engineering teams to improve scalability, reliability, observability, and developer productivity across AI systems.
• Contribute to full‑stack AI applications, developer tooling, and production deployment pipelines supporting next‑generation AI‑powered workflows.

• BS, MS, or PhD in Computer Science, Engineering, Statistics, or related technical field (or equivalent experience).
• Minimum of 2+ years of related industry experience in software engineering, AI/ML systems, distributed systems, cloud infrastructure, or generative AI applications.
• Strong programming skills in Python and/or C++ with experience building scalable software systems.
• Experience developing distributed systems, cloud infrastructure, backend services, or ML systems infrastructure.
• Hands‑on experience with machine learning frameworks such as PyTorch, Tensor Flow, JAX, or Deep Speed.
• Experience with Kubernetes, Docker, and cloud platforms such as AWS, GCP, or Azure.
• Familiarity with large language models, RAG systems, prompt engineering, evaluation frameworks, or agentic AI workflows.
• Experience building APIs and scalable services using frameworks such as FastAPI, Node.js, Type Script, or related technologies.
• Strong understanding of software engineering best practices including CI/CD, automated testing, debugging, observability, and production system reliability.

• Experience building infrastructure for distributed ML training or large‑scale inference systems.
• Background in high‑performance distributed systems, GPU scheduling, or fault‑tolerant training architectures.
• Experience developing LLM evaluation frameworks, AI safety systems, hallucination detection pipelines, or agentic AI benchmarking platforms.
• Familiarity with knowledge graphs, retrieval systems, vector databases, or scalable RAG architectures.
• Experience building Kubernetes‑based ML platforms, asynchronous evaluation systems, or cloud‑native AI infrastructure.

Base salary range: $152,000 – $241,500 (Level3) or $184,000 – $287,500 (Level4). You will also be eligible for equity and benefits.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal‑opportunity employer. We do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.