Technical Lead Manager, ML Evaluation

About Waymo

Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The World's Most Experienced Driver— to improve access to mobility while saving thousands of lives now lost to traffic crashes. The Waymo Driver powers Waymo’s fully autonomous ride-hail service and can also be applied to a range of vehicle platforms and product use cases.

The Waymo Driver has provided over ten million rider-only trips, enabled by its experience autonomously driving over 100 million miles on public roads and tens of billions in simulation across 15+ U.S. states.

The Predictive Planning team (Pre Plan) develops and deploys state-of-the-art machine learning solutions that predict the future state of the world and plan the Waymo Driver’s behavior. Our mission is to transform Waymo's unprecedented scale of driving data into robust, generalizable, and performant deep neural networks. These models enable the autonomous vehicle to navigate complex environments safely and efficiently.

We have an exciting opportunity for a Staff Technical Lead Manager to lead our ML Evaluation team
. In this role, you will define the strategic vision for our evaluation platforms, scaling the critical infrastructure and metrics required, and partner closely with the modeling teams to rigorously validate our next-generation deep neural networks and accelerate ML developer velocity across Pre Plan.

• Influence the strategic direction of foundational infrastructure and evaluation platforms to robustly support next-generation ML model evaluation use cases
• Collaborate cross-functionally with ML engineers, data scientists, and infrastructure teams to identify, define, and surface critical signals on model, component, and system-level performance
• Leverage and scale evaluation and infrastructure platforms to significantly enhance the ML developer experience, enabling faster iteration through earlier, more reliable, and trusted model evaluation
• Manage and mentor a focused team of engineers, aligning their career growth and aspirations with critical organizational needs
• Drive best practices and leverage deep technical awareness of the Alphabet ML stack (e.g.,
  Tensor Flow, JAX, Flax, Apache Beam
  ) to optimize evaluation workflows
• Stay at the forefront of emerging technologies, industry trends, and research in ML evaluation methodologies and advanced metrics design

• M.S. in Computer Science, Mathematics, or equivalent industry experience in Robotics or large-scale ML systems with critical evaluation needs
• 5+ years of experience building and maintaining large-scale distributed infrastructure, ML inference systems, or evaluation platforms, including 3+ years of engineering management experience
• Strong coding and testing proficiency, specifically in Python and C++
• Strong foundational knowledge of model evaluation and core data science principles (e.g., confidence intervals, outlier identification, curve fitting, and causality analysis)
• Familiarity with large-scale ML deployment and orchestration tools (e.g.,
  TF Serving, Torch Serve, Kubeflow, Sage Maker Pipelines, or Vertex AI Pipelines
  )
• Understanding of machine learning fundamentals and experience with popular ML frameworks such as JAX, PyTorch, or Tensor Flow

• Experience developing and maintaining evaluation pipelines for ML models
• Experience deploying and supporting machine learning models for computer vision, natural language processing, robotics/motion planning, or recommendation systems
• Experience supporting a small team of MLEs developing high-capacity, production-grade models and components
• Strong understanding of metrics computation and regression detection at scale