Machine Learning Platform Engineer, AI Evaluation

Staff Machine Learning Platform Engineer, AI Evaluation

Seattle, Washington, United States Machine Learning and AI

Join Apple Services Engineering to build the next generation of AI evaluation systems. We are seeking a staff machine learning platform engineer to lead the architectural design and development of the high availability services and internal tools powering self-service evaluation  will partner with researchers to operationalize their innovations, transforming complex workflows into intuitive, developer-first platforms. We are looking for builders who thrive in the ambiguity of new initiatives and are passionate about creating scalable infrastructure.

You will join the engineering team responsible for democratizing AI evaluation across the organization. Your focus will be on developing the developer experience—architecting and implementing the APIs, SDKs, and platform services that turn complex evaluation metrics into simple, self-service calls. You will work hand-in-hand with researchers to operationalize sophisticated measurement techniques, ensuring they scale reliably within our high-availability infrastructure. In this role, you will drive the engineering standards for a new organization, upholding the code quality, automation, and testing rigor required to support the rapid evolution of Generative AI and Agentic systems.

• System Design & Implementation:
  Lead the architecture for the core evaluation engine and distributed services, designing, coding, and shipping high-quality Python services. Own the technical direction for platform abstractions that will scale across Apple Services Engineering.
• Technical Leadership &
  
  Collaboration:
  
  Mentor engineers across the team, conduct code reviews, and drive technical decision-making. Be the person others look to when facing the hardest engineering problems, and foster a culture of technical excellence and rapid delivery through example and collaboration.
• Operationalizing Science:
  Lead the technical partnership with scientists to translate novel metrics, judge prompts, and scoring algorithms into scalable, production-grade services. Define the frameworks and architectural patterns to evaluate not just simple responses, but also multi-turn agent trajectories and tool usage.
• System Integration:
  Serve as the primary technical bridge between the research organization and the broader engineering ecosystem, driving how our tools integrate seamlessly with existing ML infrastructure and developer workflows.
• Engineering Rigor:
  Champion the software development lifecycle (SDLC) for the team, defining the standards for comprehensive automated testing (CI/CD), and instrumenting monitoring to ensure high availability and reliability.

• 8+ years of hands‑on software engineering experience, with a track record of owning the technical direction of a platform or infrastructure domain.
• Strong proficiency in the Python ecosystem (e.g., FastAPI, Pydantic, Pandas). You write production‑grade code and lead architectural discussions on day one.
• Customer Obsession & Product Thinking:
  You have owned the technical roadmap for an internal platform, presented it to senior stakeholders, and shipped against it. You independently translate vague requirements from other teams into concrete engineering specifications and platform roadmaps.
• Demonstrated experience leading technical partnerships with Data Scientists or Researchers:
  You have taken research code and shipped it as a production service and built the abstractions, testing frameworks, and deployment pipelines that made the next handoff faster than the last.
• Strong expertise in API Design & Platform Infrastructure:
  You have designed and owned APIs and SDKs that other developers rely on, with a focus on versioning, backward compatibility, and developer experience at scale.
• Operational excellence background:
  You have architected and owned CI/CD pipelines, containerization (Docker/Kubernetes), and monitoring (Datadog/Prometheus) for production services, and have been accountable for their reliability.
• Bachelors in Computer Science or related field, Masters preferred.