Principal Product Manager, Agentic Evals

Overview

Principal Product Manager, Agentic Evals within the Product Team & Core AI Experiences. Responsible for defining the AI evaluation framework to ensure safety, accuracy, and trust across Expedia Group’s AI‑powered experiences.

• Define Expedia Group’s AI evaluation framework, principles, and adoption strategy.
• Build tools and workflows that support reliable, scalable measurement across brands.
• Partner with engineering and data science on metric design, instrumentation, and model evaluation.
• Establish clear, durable standards for accuracy, safety, transparency, and performance.
• Publish evaluation insights, scorecards, and recommendations that shape roadmaps.
• Influence strategic decisions on AI investments and risk management.
• Drive alignment across product, engineering, data science, and platform teams on evaluation strategy, standards, and prioritization.
• Create governance models that ensure consistency and responsible deployment at scale.
• Shape cross‑team operating models and evaluation practices that guide how AI products are built and deployed across the company.

• Bachelor’s degree in a technical, quantitative, or related field, or equivalent practical experience.
• 10+ years of product management experience, including significant work with AI, ML, data platforms, or evaluation systems.
• Strong technical depth and experience partnering with DS/ML teams.
• Proven ability to build frameworks, metrics, or tools that scale across large organizations.
• Experience with model evaluation, instrumentation, experimentation, or system reliability.
• Strong written and verbal communication that brings clarity to complex technical spaces.
• Track record of influencing cross‑functional leaders and driving alignment.

• Experience building evaluation platforms, ML tooling, or governance frameworks.
• Familiarity with LLM behavior, model evaluation methods, or prompt testing.
• Understanding of trustworthy AI principles, safety evaluation, and failure‑mode analysis.

Pay range varies by location. See the original description for details on cash range and potential increase.

• Medical, dental, and vision coverage.
• Paid time off.
• Employee Assistance Program.
• Wellness and travel reimbursement.
• Travel discounts and International Airlines Travel Agent Network membership.

Expedia is committed to creating an inclusive work environment with a diverse workforce. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, veteran status, or any other characteristic protected by law. This employer participates in E‑Verify.