AI Governance Technology Lead

Summary

Principal, AI Testing & Monitoring is a senior technical and governance role responsible for independently validating and continuously monitoring AI/ML models to ensure they meet enterprise standards for robustness, fairness, transparency, and compliance.

• Lead the design, development, and execution of independent AI/ML model validation frameworks across various use cases.
• Conduct bias audits, adversarial testing, and stress testing to evaluate model robustness, fairness, and resilience against vulnerabilities.
• Apply statistical testing, benchmarking methodologies, and explainability (XAI) techniques to ensure models are transparent and interpretable.
• Utilize synthetic data generation and automated testing frameworks to simulate edge cases and rare scenarios for risk assessment.
• Document validation methodologies, findings, and risk-based recommendations for stakeholders, ensuring traceability and audit-readiness.
• Develop and implement enterprise AI monitoring frameworks for deployed models, focusing on real-time performance tracking, bias detection, and compliance verification.
• Apply anomaly detection and AI observability solutions to identify and remediate performance degradation, drift, or ethical risks.
• Oversee incident response for AI failures, coordinating with risk, compliance, and engineering teams to ensure timely mitigation.
• Integrate monitoring insights into governance dashboards and reporting platforms to inform executives and regulatory stakeholders.
• Ensure all testing and monitoring activities align with RUAI principles, industry best practices, and applicable regulations (e.g., EU AI Act, GDPR, CCPA, Colorado AI Act, NIST AI RMF).
• Leverage AI governance platforms and risk assessment tools to centralize validation evidence, compliance records, and ongoing monitoring metrics.
• Partner with Legal, Compliance, and Risk to interpret regulatory requirements and translate them into actionable technical and operational controls.
• Provide expert guidance to data scientists and engineers on bias mitigation, fairness optimization, and explainability best practices.
• Stay informed on emerging trends in AI risk assessment, validation methodologies, monitoring tools, and regulatory developments.
• Lead workshops, training sessions, and cross-functional knowledge sharing to advance organizational maturity in AI testing and monitoring.
• Contribute to enterprise AI governance strategy by identifying technology investments, process enhancements, and automation opportunities.
• Provide guidance and mentoring to analysts, as needed.
• Not an exhaustive list; other duties as assigned.

• Bachelor’s or Master’s degree in Computer Science, Data Science, Statistics, Information Systems, or related field.
• 5+ years in independent AI/ML model testing and validation, including robustness, fairness, and compliance verification.
• 3–5+ years in AI monitoring and risk management, including real-time model performance tracking, anomaly detection, and compliance monitoring.
• Proven experience developing and executing rigorous validation frameworks and performing bias, adversarial, and stress testing.
• Strong knowledge of AI governance principles, ethical AI frameworks, and relevant regulations (EU AI Act, GDPR, CCPA, Colorado AI Act, NIST AI RMF).
• Hands‑on experience with validation tools, statistical testing frameworks, synthetic data generation, automated testing platforms, and AI observability tools.
• Deep expertise in model validation, fairness audits, and explainability techniques.
• Proficiency in monitoring and logging frameworks for AI/ML systems.
• Strong analytical and problem‑solving skills with the ability to identify risks and propose actionable mitigations.
• Excellent written and verbal communication skills to document findings, influence stakeholders, and present to executive leadership.
• Ability to work across diverse teams and translate complex technical concepts into clear operational and compliance guidance.

• Master's Degree
• Typically, Masters in Business Administration, Computer Science, Information Management, Quantitative Analytics, Data Science, or similar discipline
• Exper…