Healthcare AI Trainer

Job Title: AI Trainer & Evaluation Specialist – Healthcare

Position Type: Contract (12 Weeks)

Hours: 10+ hours/week (Flexible schedule- you create your own schedule)

Location: Remote

We are seeking experienced clinical practitioners to evaluate how frontier AI models handle real-world clinical work.

In this role, you will bring the hard-earned judgment built co-signing a resident's note after refining the plan, catching the drug interaction an order set missed, and recognizing when a patient presentation doesn't fit the obvious diagnosis. We bring the frontier AI model outputs that require your deep clinical judgment to grade.

You will design challenging, realistic tasks drawn from your own practice—including SOAP notes, complex medication reconciliations, triage decisions, discharge summaries, patient education materials, and clinical protocols—run them through frontier AI agents, and rigorously evaluate the results against professional medical standards.

This is not a traditional clinical role
. You will directly shape the future of healthcare technology by putting your medical expertise to work in a structured, flexible, and fully remote environment. The work is long-form, analytical, and self-directed, where clear written rationale matters as much as clinical acumen.

• Design Realistic Clinical Tasks: Construct authentic clinical scenarios based on day-to-day practice. Author supporting clinical files (constructed or fully de-identified patient histories, exam findings, lab results, imaging reports, medication lists, and prior notes) and craft precise prompts framing the task as you would brief a trusted colleague.
• Evaluate AI Model Outputs: Run scenarios through frontier AI models and grade deliverables (assessments, plans, reconciliations, summaries, protocols) against the standard you would hold a professional colleague to.
• Side-by-Side Model Comparison: Compare outputs from two AI models on identical prompts and files, determine which performed better, and document precise clinical deficiencies.
• Develop Granular Grading Rubrics: Write detailed rubrics specifying what a correct deliverable must contain—such as correct differential diagnoses, red-flag identification, medication safety checks, and appropriate tone and explain why a response passes or fails.
• Identify & Flag Failure Modes: Provide evidence-based documentation for clinical errors, including unsupported assessments, missed drug interactions or contraindications, diagnostic anchoring bias, hallucinated/ignored data, unsafe recommendations, or off-brief interpretations.
• Peer Review &
  
  Collaboration:
  
  Review, refine, and stress-test task sets authored by other clinicians across various medical specialties.

• Experienced Clinician: You hold an active or previous clinical credential (e.g., MD, DO, NP, PA, PharmD, RN) with substantial hands-on experience in patient care, clinical documentation, and complex medical decision-making.
• Expert Medical Judgment: You possess strong diagnostic instincts, sharp attention to detail regarding drug safety/reimbursement/protocol compliance, and an eye for subtle clinical oversights.
• Exceptional Written Communicator: You can translate complex medical reasoning into clear, structured, written feedback and rubrics.
• Tech-Savvy & Analytical: You are comfortable working with digital tools, building structured datasets/scenarios, and working independently in a remote tech environment.
• Curious About AI: You are eager to find the limits of current AI models by crafting tasks that a competent professional would handle correctly, but that a model currently fails to navigate.

• 100% Remote & Flexible: Work from anywhere with a schedule that fits your lifestyle.
• Impactful Work: Help build safer, more reliable, and clinically sound AI tools for the future of healthcare.
• Self-Directed & Long-Form: Focus on deep, high-quality analytical evaluation rather than rapid, low-complexity tagging.