Ai Validation Analyst

Position: AI VALIDATION ANALYST
Working at Moffitt is both a career and a mission: to contribute to the prevention and cure of cancer.

As the only National Cancer Institute-designated Comprehensive Cancer Center based in Florida, Moffitt employs some of the best and brightest minds from around the world. Join a dedicated team of nearly 10,000 who are shaping the future we envision.

Moffitt has been recognized as a Best and Brightest Company to Work for in the Nation, a Digital Health Most Wired Organization and continually named one of the Tampa Bay Time's Top Workplaces.

A National Cancer Institute (NCI)-designated Comprehensive Cancer Center since 2001.

Summary

Position Highlights:

The AI Validation Analyst is the technical specialist on the AI Governance Operations team, responsible for executing local validation testing, managing continuous performance monitoring, and coordinating incident response for AI tools across Moffitt's portfolio. This role bridges the gap between clinical/operational end users and the technical realities of AI performance - translating model outputs, drift metrics, and validation results into governance-actionable findings.

The Analyst works closely with faculty validation consultants, IT/platform teams, clinical departments, and vendor technical contacts.

Responsibilities:

Local Validation Testing and Analysis

* Design and execute validation test plans for AI tools deployed or proposed for deployment at Moffitt, with emphasis on Tier 1 (high-risk) clinical AI.

* Conduct local performance validation using Moffitt data, including sensitivity/specificity analysis, subgroup performance evaluation (demographics, disease subtypes), and failure mode analysis

* Coordinate with faculty validation consultants (pathology, radiology, data science) to execute domain specific validation protocols.

* Evaluate vendor-provided validation evidence for sufficiency, including assessment of training data representativeness, external validation studies, and applicability to Moffitt's oncology patient population.

* Produce validation reports documenting methodology, findings, residual risks, and recommendations for AIGC review.

Continuous Monitoring

* Develop and manage the continuous monitoring program for deployed AI tools, including definition of performance metrics, drift detection thresholds, and alert triggers appropriate to each risk tier.

* Monitor AI tool performance for accuracy degradation, data drift, concept drift, and bias emergence using vendor-provided dashboards, internal analytics, and manual audit sampling.

* Execute scheduled review cycles aligned to risk tier (Tier 1: 6- month, Tier 2: 12-month, Tier 3: 24 month) and produce monitoring reports for AIGC.

* Identify when performance degradation or environmental changes warrant escalation, revalidation, or tool retirement.

Escalation and Incident Response

* Serve as the first responder for AI-related adverse events (aiAE), near-misses, and performance incidents.

* Coordinate incident investigation, root cause analysis, and remediation in partnership with clinical quality, patient safety, IT, and vendor contacts.

* Maintain an aiAE log integrated with Moffitt's existing quality and safety reporting systems

* Support the Director in executing emergency pause decisions when AI tools present immediate risk.

Risk Evaluation Support

* Provide technical input to the 7-question risk scoring process, particularly for questions related to clinical decision influence, human override capability, and scope of harm.

* Contribute to Tier 1 parallel specialist review tracks, specifically the Clinical Safety Track (validation requirements, failure modes, human oversight design).

* Score the residual risk factors owned by the Clinical Safety specialist track (Nature of Impact, Scale of Impact, Reversibility, Likelihood of Harm, Autonomy Level) as part of the Integrated Residual Risk Profile for Tier 1 and Tier 2 reviews; provide technical input to Data & Privacy track on Transparency factor assessments.

* Assess AI tools for equity and bias risks, particularly in oncology specific populations.

Technical Documentation and Knowledge

* Maintain technical documentation standards for validation…