Senior Product Manager, AI Harness

Who We Are Verily Health is a data platform and technology company purpose-built to power AI-enabled precision health solutions that accelerate research and improve care for individuals and communities. Uniquely positioned at the intersection of technology, data science, and healthcare, Verily transforms multimodal health data into insights, models, and actions that make healthcare more personalized, predictive, and precise.

Description Verily Health builds healthcare solutions for customers in every vertical in the market from life sciences to health systems to health plans and to patients. Each of those solutions is powered by AI capabilities. where we need to continually verify that the output is meeting the needs of our end users.

You’ll own the product strategy for Verily’s AI Evaluation Harness, the capabilities that first- and third-party AI developers use to benchmark, evaluate, monitor, and improve AI in healthcare. It spans golden-dataset evals, code assertions, LLM-as-a-judge evals, error analysis, and customer feedback signals, from quick one-off checks to long-running validation for regulated use.

You’ll set a 12-month strategy and a longer vision for what evaluation becomes in healthcare, then lead a team of software engineers and AI developers, working beside them every day, to build it. The standard you set will shape how trustworthy healthcare AI gets made at Verily and beyond. We’re looking for someone who has built evaluation products before and wants to define how the healthcare industry does it.

Responsibilities Build a platform, not a point solution. Build evaluation tools for AI developers across many customers, verticals and clinical risk levels.

Obsess about user value. Name your end users, map their journeys, deeply understand their unmet needs, and conduct continuous discovery, delivery, and iteration with your team. You are directly responsible for end to end user value creation.

Build on Platform with technical credibility. Work as a peer with software architects, engineers, and data scientists on architecture and AI tradeoffs. Align solutions with Verily’s existing platform capabilities, and work with other teams to extend those capabilities in ways that scale the platform rather than create one-offs. Align with subject matter experts across clinical, regulatory, quality, security, and data science to right-size and de-risk solutions for the business constraints we must operate within.

Own product strategy and prioritization. Develop and communicate at different levels of detail and time horizon based on the audience; 12-month product strategy, half-year milestones, quarterly delivery roadmap, and biweekly sprint priorities. Set the direction, sequence the steps, and stay hands-on with the scrum team.

Own the AI Evaluation Harness portfolio. Drive the roadmap across error analysis, golden datasets, code assertions, LLM-as-a-judge, and customer feedback signals, and align stakeholders when interests conflict.

Qualifications

Minimum Qualifications 5+ years of product management on platform products. You have been embedded with engineering scrum teams, and have a record of shipping and operating the product, not just specifying it.

Hands-on AI Evaluation product experience. You have shipped AI evaluation tooling that customers use (LLM-as-a-judge frameworks, golden datasets, code assertions and trace tooling). You can name the products, describe in detail the hard problems you solved, and show the adoption and quality outcomes for end users.

Deep technical foundation. You have a background in software engineering, systems engineering, data science, clinical informatics, or a comparable discipline. You can review architecture with engineers and go deep on LLM…