Senior Data Analyst – Agentic AI, GenAI Delivery

Responsibilities

• Validate readiness of Agentic AI use cases for production deployment
• Track deployment success metrics and post-production performance
• Identify gaps between expected vs. actual outcomes
• Define metrics for:
  Accuracy and response quality, Task completion success rates, Hallucination and failure cases, Latency and throughput
• Build evaluation datasets and validation pipelines
• Analyze:
  Agent workflows and decisions, Prompt-response chains, Tool usage and orchestration behavior
• Develop observability dashboards using telemetry and logs
• Detect and accelerate production issues and anomalies
• Perform root cause analysis on failures and performance issues
• Deliver executive‑level reporting on AI system effectiveness
• Provide actionable insights to improve system design and outcomes
• Work closely with:
  Lead Architects for feasibility alignment, AI/ML engineers for model/system improvements, Product teams for use case refinement
• Translate technical findings into clear business insights

• 6–8 years experience in data analytics, data science, or AI Systems analysis or a similar role required
• Experience supporting AI/ML or GenAI systems in production environments preferred
• Auto finance experience preferred, cross‑functional Agile team experience preferred
• Bachelor’s Degree in Data Science, Computer Science, Engineering or related quantitative field preferred
• Master’s Degree in related quantitative field preferred