Algorithm Evaluation Manager

Sunnyvale, California, United States Machine Learning and AI

We are looking for an experienced and highly motivated Engineering Manager to lead a dynamic team focused on machine learning algorithms or Large Language Model (LM) evaluation. In this role, you will guide a team responsible for the critical data and evaluation pipelines that ensure our models are accurate, robust, and performant. The ideal candidate will bring a strong mix of technical leadership, expertise in data curation and annotation processes, and deep analytical skills.

You will collaborate closely with cross-functional research and engineering teams, requiring exceptional communication and strategic thinking.

As the Engineering Manager for this team, you will be at the forefront of our AI/ML development lifecycle. Your day-to-day responsibilities will include:

• Team Leadership:
  Lead, mentor, and grow a team of engineers and data specialists. Foster a culture of innovation, rigorous analysis, and continuous learning.
• Evaluation Strategy:
  Define and execute the evaluation strategy for both CV and LM models. Build robust, scalable evaluation pipelines that accurately reflect real-world performance.
• Data Pipeline Management:
  Oversee the end-to-end data lifecycle. This includes establishing data curation guidelines, managing data quality, and optimizing large-scale annotation workflows with external vendors or internal teams.
• Analytical Deep Dives:
  Guide the team in performing rigorous data analysis to troubleshoot model regressions, uncover data quality issues, and identify opportunities for algorithmic improvements.
• Strategic Alignment:
  Act as the primary point of contact for your team, communicating progress, bottlenecks, and strategic data needs to leadership and partner teams.

• Education & Experience:
  
  BS and a minimum of 10 years relevant industry experience
• Management
  
  Experience:
  
  2+ years of direct people management experience, with a track record of hiring, mentoring, and leading high-performing technical teams.
• Domain Expertise:
  Proven experience in model evaluation, benchmarking, and A/B testing methodologies for machine learning models (Computer Vision or Foundation Models).
• Inference Infrastructure:
  Familiarity with the design and architecture of machine learning inference pipelines and underlying infrastructure.
• Data & Annotation:
  Hands-on experience designing and managing data curation strategies and human-in-the-loop annotation processes.
• Data Analysis:
  Strong analytical skills with the ability to dive deep into datasets to identify trends, biases, and areas for model improvement.
• Communication:
  Excellent verbal and written communication skills, with the ability to translate complex technical concepts to both technical and non-technical stakeholders.

• Advanced Degree:
  PhD in Computer Science, Machine Learning, or a related field.
• Deep Domain Knowledge:
  Expertise in both Computer Vision (CV) algorithms and Large Language Model (LM) evaluation methodologies (e.g., RLHF, prompt evaluation).
• Scale & Operations:
  Experience scaling large data operations, managing complex annotation workflows, and working directly with external data vendors.
• Technical Stack:
  Familiarity with Python, SQL, and ML frameworks (e.g., PyTorch) to effectively review technical work and guide engineering decisions.
• Cross-Functional Leadership:
  Demonstrated ability to drive strategic alignment across downstream product teams, ML researchers, and platform engineers in a highly matrixed environment.

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this…