Principal AI​/ML Engineer - Validation & Evaluation Governance

Why Mayo Clinic

Mayo Clinic is top-ranked in more specialties than any other care provider according to U.S. News & World Report. As we work together to put the needs of the patient first, we are also dedicated to our employees, investing in competitive compensation and comprehensive benefit plans– to take care of you and your family, now and in the future. And with continuing education and advancement opportunities at every turn, you can build a long, successful career with Mayo Clinic.

• Medical:
  
  Multiple plan options.
• Dental:
  
  Delta Dental or reimbursement account for flexible coverage.
• Vision:
  
  Affordable plan with national network.
• Pre-Tax Savings:
  
  HSA and FSAs for eligible expenses.
• Retirement:
  Competitive retirement package to secure your future.

As the Principal AI/ML Engineer - Validation & Evaluation Governance within AI Validation & Monitoring (AVM), you will serve as the enterprise subject-matter authority for validation and evaluation requirements, evidence sufficiency, and complex-case consultation. You will define and interpret standards aligned with AIA Governance Policy and patient-safety risk categories; establish retrospective and prospective pathways; set expectations for performance, safety, human-computer interaction (HCI), workflow, guardrails, production parity, material change, and revalidation;

and authoritatively review the Validation, Performance, Safety, HCI, workflow, and revalidation content and evidence for methodological adequacy, traceability, limitations, and decision readiness.

You will provide final AVM technical direction and precedent for high-risk, novel, disputed, patient-facing, agentic, vendor-limited, retrospective-method, and other precedent-setting questions. You will direct corrections, alternate methods, added evidence, interim controls, limitations, and escalation; convene cross-functional expertise; and convert recurring gaps into reusable standards and enablement.

• Providing strategic and technical leadership for enterprise validation and evaluation governance and evidence standards.
• Defining risk-proportionate requirements for pilots, full implementations, changes, legacy products, and other pathways.
• Reviewing intended use, validation strategy, performance expectations, thresholds, acceptance criteria, and residual risk.
• Evaluating test plans and datasets for representativeness, traceability, limitations, production parity, functionality, robustness, calibration, subgroup and equity evidence, and uncertainty.
• Reviewing safety controls, guardrails, task boundaries, safe refusal, escalation, fallback, human oversight, and related mitigations.
• Reviewing HCI, clinical workflow, usability, automation-bias risk, patient-facing behavior, training, accessibility, and human-use evidence.
• Determining when retrospective evidence is sufficient and when prospective testing, simulation, UAT, human-factors work, a pilot, or another method is required.
• Establishing material-change and revalidation triggers, including strengthened early monitoring when retrospective evidence replaces a pilot.
• Issuing traceable review outcomes: concurrence, edits, revision, additional evidence, consultation, escalation, or insufficient basis.
• Leading consultation and setting precedent for high-risk, novel, disputed, vendor-constrained, patient-facing, autonomous, agentic, or out-of-method cases.
• Coordinating with clinical product teams and enterprise partners, including Research Shield, Kern Center, AVSP/CCaTS, MCP Evaluate and Deploy, FAST, Epic, DTO, IT, Architecture, data platforms, Patient Safety, Clinical Informatics, Legal, and relevant committees.
• Developing policies, standards, pathway guidance, playbooks, rubrics, decision aids, evidence examples, standard findings, case libraries, and escalation criteria.
• Leading training, calibration, office hours, case consultation, quality assurance, and coaching for AVM staff and Governance Operations Product Leads.
• Translating recurring gaps into improved guidance, vendor expectations, data needs, TRex workflows, evidence objects, dashboards, and technology priorities.
• Communicating findings, limitations, uncertainty, and recommendations clearly to clinical, technical, operational, governance, and executive audiences.
• Collaborating with post-deployment governance so monitoring, reporting, significant-change, and PDRS evidence can trigger retesting, workflow evaluation, or revalidation.
• Setting precedent without owning the complete assessment,…