Senior Engineer II, AI Inference Engine Systems

Senior Engineer II, AI Inference Engine Systems

Dive in and do the best work of your career rney alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you'll find your place here. We value winning together—while learning, having fun, and making a profound difference for the dreamers and builders in the world.

Digital Ocean is expanding its AI Infrastructure layer to support the next generation of AI-driven applications. We are seeking a Senior Engineer 2 to join our AI Inference Engine Systems team. In this role, you will be a key technical leader responsible for designing, developing, and delivering high-scale, resilient data plane services that power our "Inference as a Service" offering. You will work at the intersection of distributed systems and specialized AI hardware to ensure our customers can deploy and scale their models with industry-leading performance and reliability.

This is a hands-on role, requiring you to be able to develop high quality software while availing of all the productivity boosts granted by the latest AI coding agents.

• Technical Leadership:
  Act as a technical leader on the team, driving the end-to-end design, development, and delivery of critical data plane components hosting large generative AI models.
• System Design:
  Architect and refine system design proposals for our high-scale, multi-tenant AI inference cloud ecosystem, ensuring they meet rigorous availability and resiliency standards.
• Performance Optimization:
  Implement and optimize distributed inference hosting using techniques like tensor/data parallelism, KV cache optimizations, and smart routing.
• Collaboration:
  
  Work cross-functionally with Product Managers, customer-facing teams, and other engineering teams to align technical roadmaps with customer needs.
• Mentorship:
  Coach and mentor junior engineers, fostering a culture of technical excellence and continuous improvement.
• Operational Excellence:
  Maintain and operate critical, high-scale services, utilizing observability tools and defining SLOs to ensure superior platform health.

• Distributed Systems Expertise:
  Strong experience with microservices, messaging systems, databases, and infrastructure as code.
• AI/ML Domain Knowledge:
  Hands-on experience hosting large language or multimodal models using inference engines like vLLM, SGLang, or Modular.
• Inference Frameworks:
  Familiarity with distributed inference serving frameworks such as llm-d, NVIDIA Dynamo, or Ray Serve.
• Hardware & Interconnects:
  Understanding of GPU-level optimization and experience with interconnect technologies like NVlink, XGMI, or RoCE.
• Architecture Proficiency:
  Knowledge of common LLM architectures and optimization techniques (e.g., continuous batching, quantization).
• Software Engineering:
  Expert-level proficiency in GoLang or Python and familiarity with gRPC.
• Cloud Operations:
  Proven experience shipping customer-facing software products and running critical services in a high-scale environment similar to Digital Ocean.
• Open Source Mindset:
  Experience integrating and building with open-source software.

• $167,200 - $209,000

* This is a hybrid role

• We innovate with purpose. You'll be a part of a cutting-edge technology company with an upward trajectory, who are proud to simplify cloud and AI so builders can spend more time creating software that changes the world. As a member of the team, you will be a Shark who thinks big, bold, and scrappy, like an owner with a bias for action and a powerful sense of responsibility for customers, products, employees, and decisions.
• We prioritize career development. At DO, you'll do the best work of your career. You will work with some of the smartest and most interesting people in the industry. We are a high-performance organization that will always challenge you to think big. Our organizational development team will provide you with resources to ensure you keep…