Sr. Product Mgr, AI & Human Eval Prog

About the Team And The Role

Senior Product Manager – connects category managers, merchandisers, and domain guides with scientists and engineers building AI models. Requires deep understanding of AI model development and the ability to coordinate cross‑functional work across a matrix organization.

• Run Human Evaluation and Annotation Programs – design and lead structured programs to evaluate AI model outputs across ranking, classification, generation, and extraction tasks.
• Coordinate distributed annotation teams across time zones – run workflows, maintain quality consistency, and ensure continuous coverage and on‑time delivery; handle relationships with internal and external evaluator pools.
• Translate evaluation results into clear, actionable requirements for model and product teams.
• Lead AI Quality Monitoring Programs.
• Track quality metrics, surface issues proactively, and run structured resolution processes to ensure AI‑driven experiences consistently meet the bar.
• Ensure quality signals from the live site feed back into model development priorities.
• Review what ships, ask hard questions, and hold teams accountable when AI‑driven experiences fall short.
• Help define what “good data” looks like for each focus category: coverage, freshness, labeling schema, and quality thresholds.
• Build end‑to‑end processes to meet those standards – from scoping and sourcing through annotation, QA, and delivery – in partnership with science teams and business partners.
• Define quality standards for training and evaluation data in partnership with business partners and science teams – and build the processes to meet those standards consistently.

• 5+ years of PM experience with deep exposure to AI/ML or technical platform products.
• Experience leading annotation or evaluation programs, including distributed annotator pools, external vendors, or crowdsourced labeling at scale.
• Analytical and data science fluency – comfortable writing SQL queries to explore datasets, track program metrics, and validate quality signals independently.
• Ability to work alongside Applied Scientists – understands how DS workflows operate, scopes analytical work clearly, and knows when to roll up sleeves.
• Working knowledge of how AI models are built and trained – including the role of data, human judgment, and evaluation in shaping model behavior.
• Direct exposure to LLM evaluation – preference data, benchmarking, red‑teaming, or human feedback collection is a plus.
• Experience with annotation vendors or crowdsourced labeling programs is a plus.
• Familiarity with AI observability, quality monitoring, or anomaly detection in production is a plus.

Base pay: C $133,200 - C $177,800. Base pay may vary based on location, skills, and experience. Total compensation may include target bonus and restricted stock units (as applicable). Additional benefits include a full range of medical, financial, and other benefits (including RRSP eligibility, paid time off such as PTO and parental leave).

eBay is an equal chance employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, disability, or other legally protected status. If you have a need that requires accommodation, please contact us at  We will make every effort to respond to your request for accommodation as soon as possible.