Lead Quality Automation Engineer – AI Platform

We are seeking a Senior Test Automation Engineer & Quality Engineering Lead to establish and scale quality engineering for a modern AI platform and AI-enabled applications.

This is a hands-on, player-coach role responsible for defining the automation strategy, building reusable frameworks and critical test suites, integrating quality into CI/CD, and helping engineering teams increasingly own product quality.

The role spans web and mobile applications, APIs and backend services, data and integrations, and AI/agent behavior.

• Define an automation-first strategy across UI, APIs, services, integrations, data, and AI workflows.
• Build reusable automation frameworks and high-value regression suites, with API/service-level testing as the foundation
  .
• Automate critical web, mobile, and end-to-end journeys, including authentication, authorization, multi-tenant behavior, asynchronous interactions, and failure scenarios.
• Integrate automated tests and quality gates into CI/CD pipelines.
• Establish test execution monitoring, failure triage, reporting, and flaky-test management.
• Automate critical performance, latency, resilience, concurrency, and dependency-failure scenarios.

• Establish repeatable evaluation and regression testing for Generative AI and agentic applications.
• Validate appropriate tool and data usage, workflow and authorization boundaries, human-approval controls, and handling of missing or conflicting information.
• Validate groundedness, factual accuracy, relevance, evidence quality, hallucination risk, instruction adherence, latency, and reliability.
• Test retrieval, tool-calling, multi-step agent workflows, and AI application failure scenarios.
• Automate data-quality validation covering freshness, integrity, completeness, transformation, traceability, and tenant isolation.
• Establish representative synthetic and simulated datasets for deterministic regression and AI evaluation.

• Build the initial automation architecture and highest-value test suites.
• Lead automation design, code quality, and engineering standards.
• Partner with software, platform, data, AI, and Dev Ops engineers to embed testability early.
• Participate in architecture and design reviews and mentor engineers contributing to shared automation frameworks.
• Promote the right mix of unit, API, integration, end-to-end, performance, exploratory, and AI evaluation testing while driving distributed quality ownership.

Strong candidates will have:

• Senior-level test automation or quality engineering experience, including designing automation frameworks from the ground up.
• Strong Python skills for API, service, data, and AI test automation.
• Strong web, API/service, integration, and end-to-end automation experience.
• Experience with automated testing of iOS and Android applications; cross-platform mobile automation is desirable.
• Strong experience testing REST APIs
  , asynchronous services, authentication, authorization, and multi-tenant systems.
• Strong SQL and data-validation skills across relational, operational, time-series, and document-oriented data.
• Hands-on experience testing applications and services deployed on AWS
  ; AWS application, data, container, and observability services are strongly preferred.
• Experience testing applications built with OpenAI models/APIs or comparable production LLM platforms;
  OpenAI experience preferred.
• Hands-on experience with Generative AI and agentic systems
  , including retrieval, grounding, tool-calling, multi-step workflows, behavioral regression, hallucination/evidence validation, and AI observability.
• Experience integrating automated testing into Git-based CI/CD workflows;
  Bitbucket and Bitbucket Pipelines experience preferred.
• Experience…