Senior Data Scientist; AI Metrics & Portal

Position Overview

The Data Scientist, AI Metrics & Portal is a technical role responsible for owning the full lifecycle of AI Program metrics, including defining, architecting, implementing, operationalizing, and continuously improving a standardized AI metrics capability. This role combines data science, analytics engineering, artificial intelligence, and software development to:

• Establish AI Program metrics—from conceptual definition through technical implementation and ongoing optimization.
• Design, build, and operate a modern, lightweight AI Metrics Hub, leveraging Claude Code and other technology stack tools to rapidly develop and maintain an extensible analytics platform.

The Data Scientist will define and operationalize standardized AI metrics, architect the supporting data and application layers, implement dynamic visualization and AI‑driven querying capabilities, and ensure continuous evolution of the platform to meet business needs.

The role will orchestrate metrics design, platform engineering, and Agile delivery practices to:

• Define, standardize, and govern AI metrics across adoption, utilization, performance, value, cost, risk, and other categories.
• Architect scalable data models and metrics frameworks to ensure consistency and reuse.
• Implement and operationalize metrics pipelines, logic, and computation layers.
• Design and build an analytics platform with AI metrics catalog, standard/pre‑configured AI dashboards, and self‑service AI dashboards and exploration.
• Implement AI‑powered natural language querying and discovery capabilities.
• Maintain and evolve metrics definitions, lineage, and supporting documentation.
• Deliver iteratively using Agile and SAFe methodologies.
• Enable continuous improvement and future integration with enterprise platforms (e.g., Databricks, Collibra).

This role requires a balance of hands‑on implementation, architecture ownership, and delivery leadership, with accountability for the end‑to‑end lifecycle of AI metrics and insights capabilities.

• 1. AI Metrics Lifecycle Ownership (Define Architect Implement Operate Evolve)
  • Own the full lifecycle of AI metrics, including:
    • Definition and standardization
    • Architectural design
    • Technical implementation
    • Operational monitoring
    • Continuous improvement
  • Define and maintain a comprehensive AI metrics framework, including:
    • Adoption, utilization, engagement
    • Business value and ROI
    • Performance and quality
    • Risk, compliance, and cost
  • Translate business questions into well‑defined, implementable metrics and models
• 2. Metrics Architecture & Standardization
  • Architect scalable, reusable metric models, including:
    • KPI definitions and calculation logic
    • Dimensional structures and aggregation strategies
  • Establish and enforce standards for consistency, governance, and reuse
  • Ensure metrics are designed for extensibility and enterprise integration
• 3. Metrics Implementation & Data Engineering
  • Design and implement metrics computation pipelines and transformations
  • Develop and maintain SQL and Python logic for KPI calculation
  • Integrate and normalize data from multiple sources (logs, APIs, databases, surveys, risk reviews, and more)
  • Ensure data accuracy, consistency, and performance optimization
  • Implement data quality validation and monitoring processes
• 4. AI Metrics Portal Development
  • Architect, build, and maintain the AI Metrics Hub application
  • Develop platform components, including:
    • Metrics registry (definitions, metadata, ownership)
    • Dynamic dashboard and visualization engine
    • Config‑driven metric execution layer
  • Leverage AI‑assisted development tools (e.g., Claude Code) to:
    • Accelerate development
    • Generate reusable assets
    • Improve maintainability
  • Ensure platform supports rapid iteration and long‑term scalability
• 5. AI / NLP / RAG Integration
  • Design and implement natural language interfaces for interacting with metrics
  • Build and maintain RAG pipelines leveraging:
    • Metric definitions
    • Metadata and contextual information
  • Develop prompt engineering strategies and query translation logic
  • Enable workflows such as:
    Ask a question → generate query → return visualization and explanation
  • Continuously improve AI output accuracy, usability, and relevance
• 6. Visualization & Self‑Service…