Skip to content
×

Machine Learning Evaluation Engineer

Listing for: Apple Inc.
Full Time position Listed on 2026-10-18
Job specializations:
Salary/Wage Range or Industry Benchmark: 150400 - 277600 USD Yearly USD 150400.00 277600.00 YEAR

Job Description & How to Apply Below

The Data, Analytics and Quality (DAQ) team’s core mission is to evaluate and elevate advanced sensing technologies. We collaborate closely with partners in computer vision, video engineering, and other applied ML domains to deliver high quality algorithms that power intelligent features across Apple’s products. Our evaluations and analyses inform these teams and Apple leadership throughout the entire development cycle, from early prototyping to shipping a refined product.

We are seeking a Machine Learning Evaluation Engineer who will combine deep technical expertise, creativity, and systems thinking to design and deploy new evaluation strategies for multi-modal foundation models and task-specific CV/ML algorithms.

Description

As a key member of this team, you will lead the benchmarking of state-of-the‑art multi‑modal models, developing comprehensive evaluation systems that include metric design, large‑scale data preparation, and advanced failure analysis automation. Rather than focusing on core model training, you will apply your deep CV and ML expertise to rigorously assess model capabilities and translate findings into actionable improvements. You will leverage modern GenAI tools to improve engineering efficiency and accelerate analysis.

You will collaborate with multidisciplinary teams across HW, SW, design, and other applied fields to ensure models meet the high quality bar necessary for exceptional customer experiences. In this role you'll be responsible for:

Algorithm Evaluation & Benchmarking:
Design and build comprehensive evaluation pipelines that scale across large datasets. You will be responsible for both holistic end‑to‑end system evaluation and granular component‑level testing to rigorously measure model capabilities on complex text, image, and video understanding tasks.

Subjective Evaluation:
Solve the challenge of developing and calibrating subjective evaluations for generative models by leveraging LLM-as-Judge and human grading techniques.

Deep Failure Analysis:
Identify trends in large datasets and dive deep into specific failure cases to find their root causes in models, prompts, or upstream software and algorithm components. Develop failure taxonomies to track over time.

Workflow Automation:
Design and develop cloud‑based, LLM‑powered workflows and dashboards to streamline failure analysis, reporting, and iterative experimentation. This is key to efficiently isolate and triage problems within complex multi‑stage algorithm stacks.

Evaluation Set Curation:
Refine our strategy for curating high‑quality datasets and ground truth, whether through auto‑labeling with VLMs, leveraging manual annotation teams, or generating synthetic data.

Data Analysis: assess quality, diversity, representativeness, and coverage gaps across large, complex datasets to ensure evaluation sets reflect real‑world usage.

Cross‑Functional

Collaboration:

Partner closely with the core model training teams. You will provide them with actionable, data‑driven insights and metrics to guide the next iteration of model training and fine‑tuning.

Minimum Qualifications
  • BS and a minimum of 3 years relevant industry experience
  • 3+ years of applied experience in Machine Learning, Computer Vision, or AI System Evaluation.
  • Deep understanding of core Machine Learning principles, including probability, statistics, data distributions, and model bias/variance. Ability to apply statistical rigor to ensure evaluation metrics are meaningful and reliable.
  • Experience analyzing data quality, diversity, representativeness, and coverage gaps in large datasets, and translating findings into data collection and curation strategies.
  • Applied experience employing and evaluating generative models, including knowledge of prompt tuning,…
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction

Search for further Jobs Here:

SEARCH JOBS: 10+ Million Listings
(Try combinations for better Results! Or enter less keywords for broader Results)
Education Level
Experience Level (years)
Posted in last:
Salary