Senior Consultant – Ai Product​/Business Analyst – Ux Engineering

Senior Consultant – AI Product / Business Analyst – UX Engineering

Hartford Long Finch Technologies

Posted 10 days ago

Job Summary

We are seeking an experienced
Senior Consultant – AI Product / Business Analyst to support the development and validation of
AI-powered products and solutions . The ideal candidate will have strong experience working as a
Business Analyst, Product Analyst, Product Owner, or similar role , along with hands‑on experience in
AI/LLM evaluation, prompt engineering, AI testing, and product validation .

The candidate will work closely with
Business Stakeholders, Engineering, UX, and Data Science teams to translate business and AI requirements into actionable product features, user stories, workflows, and acceptance criteria. Experience in the
Healthcare domain is highly preferred, along with hands‑on AI experience.

Key Responsibilities

• Gather and analyze business requirements for AI-powered products and solutions.
• Translate business requirements into detailed
  Jira user stories, acceptance criteria, workflows, and product requirements .
• Manage and prioritize product backlogs in collaboration with Product Owners and stakeholders.
• Design and execute specialized
  AI/LLM test cases and evaluation scenarios .
• Validate LLM-generated outputs for
  accuracy, quality, completeness, relevance, consistency, and reliability .
• Evaluate different prompt variations and identify opportunities for prompt optimization.
• Develop and execute test scenarios for
  AI agents and agentic workflows .
• Perform AI model/product validation and analyze AI performance metrics.
• Identify defects, inconsistencies, and quality gaps in AI-generated content and workflows.
• Collaborate with Engineering, UX, Data Science, and Business teams to define and prioritize AI capabilities.
• Support Agile ceremonies including backlog refinement, sprint planning, reviews, and retrospectives.
• Drive continuous improvement of AI product quality and user experience.
• Document requirements, test results, evaluation findings, and product recommendations.
• Partner with stakeholders to ensure AI solutions meet business objectives and user needs.

Required Skills & Qualifications

• 8–10+ years of experience as a Business Analyst, Product Analyst, Product Owner, Product Consultant, or equivalent role.
• Hands‑on experience working with
  AI-powered products, Generative AI, LLMs, or AI/ML solutions .
• Strong experience creating
  Jira user stories, acceptance criteria, product requirements, and workflows .
• Hands‑on experience with
  AI/LLM testing and evaluation .
• Experience evaluating LLM outputs and comparing prompt variations.
• Strong understanding of
  prompt engineering and prompt optimization .
• Experience validating AI-generated content for accuracy, completeness, consistency, and quality.
• Experience testing or evaluating
  AI agents and agentic workflows .
• Excellent communication and stakeholder management skills.
• Ability to work effectively with
  Engineering, UX, Data Science, Product, and Business teams .

Preferred / Nice-to-Have Skills

• Healthcare domain experience is highly preferred.
• Experience in
  HR technology, learning technology, content creation platforms, or knowledge management solutions .
• Experience working with AI-powered enterprise applications.
• Knowledge of AI evaluation methodologies and performance metrics.
• Experience with Jira and Agile product development environments.
• Experience contributing to UX/product discovery and user journey analysis.

Top 3 Responsibilities

• Requirements & Product Management: Gather business requirements, create detailed Jira user stories, define acceptance criteria, and manage product backlogs while translating AI product requirements into actionable features and workflows.
• AI/LLM Evaluation & Testing: Plan, execute, and monitor AI/LLM evaluations by designing test scenarios, validating model outputs, evaluating prompts and agentic workflows, analyzing metrics, and driving continuous product quality improvements.
• Cross-Functional Collaboration: Collaborate with
  Engineering, UX, Data Science, Product, and Business stakeholders to prioritize AI capabilities and ensure successful delivery of AI-powered products.