Sr. AI​/ML QA Automation Engineer

Cupertino, California, United States Software and Services

The Apple Photos application is a comprehensive photo and video management solution that seamlessly integrates across the entire Apple ecosystem, enabling users to capture, organize, edit, and share their visual memories with unprecedented ease and intelligence. Working on the Photos team means contributing to one of Apple's most personal and widely-used applications, combining cutting-edge AI, elegant design, and robust engineering to help billions of users around the world preserve and relive their most important life moments.

This is a high-impact role where you'll work at the intersection of AI modeling, agentic workflows, information retrieval, software engineering, evaluation and metrics, and help us push the boundaries of how AI can transform Apple’s products.

This role blends traditional software QA skills with advanced evaluation methodologies for modern AI models, including LLMs, multimodal systems, and ML-driven product features. As a member of the team, you will work closely with experienced engineers and machine learning experts to qualify and refine features powered by vision, language, and cross-modal intelligence. You will be responsible for designing rigorous evaluation strategies for both objective and subjective ML behaviors, creating reliable automated testing pipelines, and developing LLM-driven evaluators that complement human judgement.

The ideal candidate is self-directed, creative, and comfortable with ambiguity, with strong technical and interpersonal skills. They have hands‑on experience testing ML models directly, defining qualitative scoring rubrics, building reproducible evaluation frameworks, and ensuring that AI behavior is safe, consistent, and aligned with product and on‑device constraints.

• Design and implement test strategies for qualifying subjective and ML‑driven features
• Perform Functional testing of existing/new features on various platforms across concurrent release vectors
• Design and execute evaluation plans for multimodal ML models (vision, language, and cross‑modal systems)
• Develop and maintain robust testing frameworks for machine learning models
• Collaborate with QA teams to integrate models into existing test frameworks
• Consider on‑device ML performance constraints (latency, memory, energy) when designing tests and evaluation strategies

• BS/MS or equivalent experience in Computer Science or related field
• 3+ years of experience working in Software Quality Assurance
• Strong software engineering skills, including system design, development, testing, debugging, release and maintenance
• Expertise with hands‑on experience in automated software testing, data validation, and ethical AI practices
• Familiarity with LLM usage to improve efficiency of their daily work
• Ability to evaluate ML models directly (vision or multimodal) and diagnose model failures using quantitative and qualitative methods
• Expertise in Python
• Proficiency with iOS, macOS, watch
  
  OS, tvOS or similar operating systems

• Experience Testing AI Models for accuracy, robustness, fairness, and performance
• Experience developing LLM based automated evaluation frameworks
• Expertise in Swift and/or Obj‑C
• Strong programming skills in Python and experience with ML/NLP libraries
• 3+ years of proven ability in machine learning, including hands‑on work with LLMs
• Understanding of prompt engineering, and retrieval‑augmented generation (RAG)
• Knowledge of statistics based evaluation approaches, ML training pipelines and accuracy improvements of ML systems

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $147,400 and $272,100, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and…