Applied Science Manager, Artificial General Intelligence

Overview

The Artificial General Intelligence (AGI) team is seeking a dedicated, skilled, and innovative Applied Science Manager with a robust background in machine learning, statistics, quality assurance, auditing methodologies, and automated evaluation systems to lead a team ensuring the highest standards of data quality, to build industry-leading technology with Large Language Models (LLMs) and multimodal systems.

• Lead and mentor a team of Applied Scientists who develop comprehensive quality strategies and auditing frameworks that safeguard the integrity of data collection workflows.
• Design auditing strategies with detailed SOPs, quality metrics, and sampling methodologies aligned with the core scientist team developing Amazon Nova models.
• Oversee expert-level manual audits, meta-audits to evaluate auditor performance, and provide coaching to uplift overall quality capabilities across the team.
• Manage the development and maintenance of LLM-as-a-Judge systems, including designing judge architectures, creating evaluation rubrics, and building machine learning models for automated quality assessment.
• Oversee the configuration of data collection workflows and ensure effective communication of quality feedback to stakeholders.
• Direct impact on customer experiences through high-quality training and evaluation data powering state-of-the-art LLM products and services.
• Recruit, hire, and develop team members; conduct performance reviews; set clear expectations and growth plans; and foster a culture of scientific excellence and innovation.
• Communicate with senior leadership, cross-functional technical teams, and customers to collect requirements, describe product features and technical designs, and articulate product strategy.
• Lead quality solution design, guide root cause analyses on data quality issues, drive research into new auditing methodologies, and optimize data quality while exemplifying QA best practices.
• Collaborate with engineers, domain experts, and vendor teams to implement quality strategies and automated judging systems; conduct regular 1:1s and mentorship to ensure high-impact results aligned with organizational goals.

• 2+ years of team management experience
• Master's degree in engineering, technology, computer science, machine learning, robotics, operations research, statistics, mathematics or equivalent quantitative field
• 2+ years of machine learning, statistical modeling, data mining, and analytics techniques experience
• 2+ years of building machine learning models or developing algorithms for business applications
• 3+ years of programming in Java, C++, Python or related language

• Experience communicating with users, other technical teams, and management to collect requirements, describe software product features, and technical designs
• 3+ years of scientists or machine learning engineers management experience
• Ph.D. in engineering, technology, computer science, machine learning, robotics, operations research, statistics, mathematics or equivalent quantitative field
• Publications at top-tier peer-reviewed conferences or journals
• 4+ years of solving business problems through machine learning, data mining and statistical algorithms experience
• Experience with Machine Learning and Large Language Model fundamentals, including architecture, training/inference life cycles, and optimization of model execution, or experience in developing and deploying LLMs in production on GPUs, Neuron, TPU or other AI acceleration hardware

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit (Use the "Apply for this Job" box below). for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave.

Learn more about our benefits at .

USA, MA, Boston -  -  USD annually

USA, WA, Bellevue -  -  USD annually

Services LLC

Job : A3162391