Senior Specialist, Quality​/M&P​/Process – AI Training Manager

Responsibilities

• Monitor the performance of Agentic Capabilities as they autonomously process various ticket types.
• Ensure seamless integration of AI agents into new or existing workflows, optimizing for efficiency and accuracy.
• Review fallout tickets (cases where AI agents cannot resolve issues) within a workflow management tool.
• Diagnose root causes, make necessary corrections, and re‑ingest updated information to the AI system.
• Ensure all fallout tickets are actioned within a 48‑hour window; unresolved tickets revert to the human‑worked queue.
• Analyze fallout patterns to identify knowledge gaps, process inefficiencies, or opportunities for AI improvement.
• Develop and implement training protocols to enhance Agentic Capabilities, leveraging prompt engineering, model validation, and knowledge base updates.
• Collaborate with cross‑functional teams (product, engineering, support) to align AI behaviors with business needs and compliance requirements.
• Maintain and update the agent knowledge base, ensuring accurate, current, and comprehensive content for AI agents.
• Document training methodologies, ticket resolutions, and process improvements for knowledge sharing and auditing.
• Validate AI performance through systematic review, testing, and user/stakeholder feedback.
• Ensure all processes comply with regulatory standards, ethical guidelines, and company policies.
• Track and report on key metrics: ticket resolution rates, fallout frequency, review turnaround times, and AI improvement outcomes.
• Communicate insights, best practices, and recommendations to stakeholders and leadership.

• Understanding of the business function and M&Ps; align AI behavior with policy and process; plus knowledge of supported workflows and tools with experience operating the systems involved (e.g., ticketing/workflow platforms).
• Strong analytical and problem‑solving skills with meticulous attention to detail; proven root cause analysis (RCA) capability.
• General AI literacy and understanding of agentic systems; basic prompt engineering (iteration, testing, versioning).
• Ability to manage fallout within SLAs, triage tickets, and drive rapid resolution; strong prioritization in fast‑paced environments.
• Observability: proficiency with logs, metrics, dashboards, and alerts; define and track quality KPIs (accuracy, fallout rate, MTTR).
• Basic scripting understanding to automate corrections, content re‑ingestion, and validation workflows.
• Knowledge base authoring and maintenance; clear documentation of training methods, resolutions, and changes for auditability.
• Compliance/data privacy/ethical guidelines awareness; maintain auditable processes and change logs.
• Effective communication: synthesize findings, report metrics, and present recommendations to stakeholders.