Expert & AI Validator; Remote Contractor
Burlington, Ontario, A9A, Canada
Listed on 2026-10-20
-
Business
AI Evaluation
About League
League is one of the fastest-growing technology companies in Canada and the leading healthcare experience platform. Getting healthcare is often the easy part — finishing it is where things fall apart: people book the appointment and skip the follow-up, fill the prescription and stop taking it, get the referral and never make the call. That gap costs health plans and health systems money, and it costs people their health.
League closes that gap — identifying what each person needs to do next, clearing what’s in their way, and getting it done, for the 70 million+ people whose care already runs through our platform. Health plans and health systems trust us to do this anizations like Manulife, SCAN, Geisinger, and Medibank on the payer side, and Baptist and Shoppers Drug Mart on the provider side.
About League
League is one of the fastest-growing technology companies in Canada and the leading healthcare experience platform. Getting healthcare is often the easy part — finishing it is where things fall apart: people book the appointment and skip the follow-up, fill the prescription and stop taking it, get the referral and never make the call. That gap costs health plans and health systems money, and it costs people their health.
League closes that gap — identifying what each person needs to do next, clearing what’s in their way, and getting it done, for the 70 million+ people whose care already runs through our platform. Health plans and health systems trust us to do this anizations like Manulife, SCAN, Geisinger, and Medibank on the payer side, and Baptist and Shoppers Drug Mart on the provider side.
Position Summary
League is building the infrastructure behind our Benefits Navigation Agent — the AI that helps members understand and use their health benefits — and we're growing our panel of expert reviewers. The through-line is domain judgment: the situational, real-world reasoning that determines the guidance of our agent gives members is accurate, complete, and trustworthy.
You'll help validate the rubrics we use to assess the agent's answers, grade responses against accuracy and coverage standards, and pressure-test the scenarios that keep benefits guidance correct and trustworthy. This is a great fit for benefits and insurance professionals who want flexible, remote work — and who want their expertise to shape a product while it's still being built.
The Kind Of Work You Might Do
Engagements vary, but could include:
- Scenario design and pressure-testing — building and stress-testing the member scenarios the agent is evaluated against, so the rubric reflects the questions members actually ask and the failure modes that matter most.
- AI agent evaluation — supporting the development and validation of rubrics, grading AI-generated benefits guidance against structured accuracy and coverage standards, flagging where the agent misstates coverage or oversteps into advice it shouldn't give, and providing written feedback on how responses can be improved.
- Subject-matter input — bringing a benefits, claims, or regulatory perspective to how we define correctness, coverage, and the hard-fail boundaries the agent must never cross.
We're building a multidisciplinary panel. You don't need to cover all of these — we're looking for depth in one or more of the following areas:
- Benefits plan design and administration — You validate that the agent correctly interprets a member's plan documents rather than plausibly paraphrasing them wrong. This is the foundation of every coverage answer the agent gives.
- Payer / claims operations and live-data interpretation — You ensure the agent doesn't just echo API numbers but understands what they actually mean, catching failure modes…
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).