Senior Software Engineer, Inference Platform

We’re looking for a Senior Engineer to help build the next-generation inference platform that supports embedding models used for semantic search, retrieval, and AI-native experiences in MongoDB Atlas. You’ll join the broader Search and AI Platform organization, collaborate with ML researchers and engineers from our Voyage.ai acquisition, and build infrastructure for real-time, low-latency, high-scale inference fully integrated with Atlas, designed for developer-first experiences.

We're looking to speak with candidates in Palo Alto for our hybrid working model.

• Design and build components of a multi-tenant inference platform integrated directly with MongoDB Atlas, supporting semantic search and hybrid retrieval
• Collaborate with AI engineers and researchers to product ionize inference for embedding models and rerankers — enabling both batch and real-time use cases
• Contribute to platform capabilities such as latency-aware routing, model versioning, health monitoring, and observability
• Improve performance, autoscaling, GPU utilization, and resource efficiency in a cloud-native environment
• Work across product, infrastructure, and ML teams to ensure the inference platform meets the scale, reliability, and latency demands of Atlas users
• Gain hands‑on experience with tools like vLLM and container orchestration with Kubernetes

• 5+ years of experience building backend or infrastructure systems at scale
• Strong software engineering skills in languages such as Go, Rust, Python, or C++, with an emphasis on performance and reliability
• Experienced in cloud-native architectures, distributed systems, and multi-tenant service design
• Familiar with concepts in ML model serving and inference runtimes, even if not directly deploying models
• Knowledge of vector search systems (e.g., Faiss, HNSW, ScaNN) is a plus
• Comfortable working across functional teams, including ML researchers, backend engineers, and platform teams
• Motivated to work on systems integrated into MongoDB Atlas and used by thousands of developers

• Experience integrating infrastructure with production ML workloads
• Understanding of hybrid retrieval, prompt-driven systems, or retrieval-augmented generation (RAG)
• Contributions to open-source infrastructure for ML serving or search

• Be part of building the AI foundation of the world’s most popular developer data platform
• Collaborate with ML researchers from Voyage.ai to bring novel ideas into scalable systems
• Tackle challenging problems in inference, observability, and distributed infrastructure
• Work in a culture that emphasizes mentorship, ownership, and technical excellence

MongoDB’s base salary range for this role in the U.S. is: $126,000-$248,000 USD.

Other benefits for eligible employees may include equity, participation in the employee stock purchase program, flexible paid time off, fully-paid gender-neutral parental leave, fertility and adoption assistance, 401(k) plan, mental health counseling, transgender-inclusive health insurance coverage, and standard health benefits.

MongoDB, Inc. provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type and makes all hiring decisions without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

Req :