Senior AI​/ML Engineer - Validation & Evaluation

Why Mayo Clinic

Mayo Clinic is top-ranked in more specialties than any other care provider according to U.S. News & World Report. As we work together to put the needs of the patient first, we are also dedicated to our employees, investing in competitive compensation and comprehensive benefit plans – to take care of you and your family, now and in the future. And with continuing education and advancement opportunities at every turn, you can build a long, successful career with Mayo Clinic.

City Rochester

State MN

Remote NO

Department Information Technology

Mayo Clinic is top-ranked in more specialties than any other care provider according to U.S. News & World Report. As we work together to put the needs of the patient first, we are also dedicated to our employees, investing in competitive compensation and comprehensive benefit plans – to take care of you and your family, now and in the future. And with continuing education and advancement opportunities at every turn, you can build a long, successful career with Mayo Clinic.

• Medical:
  Multiple plan options.
• Dental:
  Delta Dental or reimbursement account for flexible coverage.
• Vision:
  Affordable plan with national network.
• Pre-Tax Savings: HSA and FSAs for eligible expenses.
• Retirement:
  Competitive retirement package to secure your future.

As the Senior AI/ML Engineer - Validation & Evaluation within AI Validation & Monitoring (AVM), you will provide practice leadership for validation pathways, applied evaluation, and consultation. You will translate approved enterprise requirements into practical validation and evaluation methods; lead the development and maintenance of guidance for human-computer interaction (HCI), usability, clinical workflow, human oversight, pilot HCI design, legacy-product HCI remediation, and revalidation;

and review Validation, Performance, and Safety content for methodological adequacy, evidence sufficiency, consistency, limitations, and alignment with approved policy and methods.

You will lead complex but generally non-precedent-setting assessment review and consultation, calibrate AVM reviewer comments, assure the quality and consistency of review conclusions and revalidation guidance, and coach AVM Engineers, Associates, and others in the department. You will recommend required corrections, alternate methods, additional evidence, limitations, fallback or remediation strategies, and escalation.

• Translating approved enterprise validation and evaluation requirements into practical pathways and applied methods for pilots, full implementations, post-deployment changes, legacy products, and other governed use cases.
• Reviewing intended use, pathway selection, performance and safety expectations, standard-of-practice comparisons, acceptance thresholds, evaluation criteria, evidence sufficiency, limitations, and residual uncertainty.
• Evaluating the adequacy of retrospective studies, prospective designs, pilot protocols, workflow simulations, user acceptance testing, human-factors work, and alternative validation strategies.
• Reviewing test plans/protocols, methods and validation datasets for representativeness, traceability, production parity, functionality, robustness, calibration, subgroup and equity evidence, uncertainty, and methodological limitations.
• Reviewing HCI, clinical workflow, usability, human oversight, automation-bias risk, patient-facing behavior, training, accessibility, guardrails, task boundaries, safe refusal, escalation, fallback, and remediation strategies.
• Leading complex case consultation and resolving non-precedent-setting method questions within approved standards, while escalating novel, precedent-setting, disputed, or out-of-method questions to the Principal…