Machine Learning, Platform Engineer

About the Role

Our team focuses on enabling custom models and dedicated inference on Together. We are responsible for building a container platform, optimizing autoscaling, minimizing cold starts, achieving the best end-to-end model performance, and providing a best-in-class developer experience with great tooling. We often focus on video or audio generation across the stack:
CUDA kernels
, pytorch optimization
, inference engines
, container orchestration
, queueing theory
, etc. An ideal candidate will be great at profiling/optimization but know the word kubernetes
, or be intimately familiar with multi-cluster scheduling and have some sense of ML bottlenecks.

• New hires may work on multi-cluster orchestration, portfolio optimization, predictive autoscaling, control panes, model bring-up, model optimization, APIs for managing deployments, inference worker SDKs, and CLI tools.
• Analyze and improve the robustness and scalability of existing distributed systems, APIs, databases, and infrastructure
• Partner with product teams to understand functional requirements and deliver solutions that meet business needs
• Write clear, well-tested, and maintainable software and IaC for both new and existing systems
• Conduct design and code reviews, create developer documentation, and develop testing strategies for robustness and fault tolerance

• 5+ years of demonstrated experience in building large scale, fault tolerant, distributed systems.
• Experience running serverless inference platforms, doing model bring-up on short notice, being on call, or running a cloud provider is a very big plus
• Good taste and ability to thoughtfully discuss how what you’ve built has failed over time
• Experience designing, analyzing and improving efficiency, scalability, and stability of various system resources
• Excellent understanding of low level operating systems concepts including concurrency, networking and storage, performance and scale
• Expert-level programmer in one or more of Python
  , Golang
  , Rust
  , C++, or Haskell
• Proficiency in writing and maintaining Infrastructure as Code (IaC) using tools like Terraform
• Experience with Kubernetes internals or other container orchestration systems
• Sound judgement for when to use and when to not use LLMs for code
• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related technical field, or equivalent practical experience
• Writing-heavy roles or companies are a plus