QA Lead - AI​/ML

E-commerce domain exp. is mandatory.
Job Description:
The AI/ML Test Lead will own end to end quality assurance for AI driven solutions across Ulta's Enterprise AI.This role is responsible for defining and executing an Enterprise AI testing strategy
, ensuring functional correctness, data accuracy, AI output quality, robustness, performance, and Responsible AI compliance
.
The role blends hands on testing leadership
, AI/ML specific validation
, and onsite stakeholder collaboration
, working closely with Product Owners, AI Engineering Leads and Data teams.
Required Skills & Experience-
Core QA & Testing Experience

• 10+ years of experience in QA / Testing roles
  , with leadership responsibility
• Strong experience defining test strategy, test plans, and execution ownership
• Experience testing data heavy and analytics driven systems

• Hands on experience testing AI/ML or LLM based solutions
• Strong understanding of:
  • Data validation and KPI testing
  • AI output correctness and consistency
  • Prompt based and agent based systems
• Familiarity with AI evaluation metrics (accuracy, error rate, hallucination rate, relevance)