Senior QA Automation Engineer – AI

Position: Senior QA Automation Engineer – AI Applications
** Senior QA Automation Engineer – AI Applications
** We are seeking an experienced QA Automation Engineer to ensure the quality, reliability, and performance of AI-powered applications. The ideal candidate has expertise in automated testing and software quality engineering, with experience building scalable testing frameworks and leveraging AI-powered testing tools to improve test development, maintenance, and coverage. They combine deep QA automation expertise with practical experience using AI to enhance testing workflows.

Experience testing AI/ML applications or validating machine learning models is an asset.
** Responsibilities
* ** Design and maintain automated test suites, evaluation scenarios, metrics, and test datasets for AI-powered features.
* Automate QA workflows using Python and CI/CD (e.g., Git Hub Actions, Jenkins), leveraging AI-powered testing tools to improve test development, maintenance, and coverage.
* Create and maintain test harnesses for AI services, applications, and APIs.
* Perform API testing for AI services and backend systems.
* Define quality strategy and acceptance criteria for AI-powered applications across functional, API, integration, and AI/ML features.
* Test AI/ML applications and machine learning models from a product and user perspective.
* Evaluate AI/ML model outputs for correctness, consistency, and expected application behavior.
* Define structured rubrics that translate qualitative findings into measurable evaluation gates.
* Build quality gates for CI/CD and deployment pipelines, including automated regression and integration testing.
* Mentor teams on QA automation best practices and consistent quality standards for AI-powered applications.
* Contribute to multi-team projects and codebases, ensuring code quality and consistency.
* Participate in code reviews and provide constructive feedback to peers.
* Document and present findings and ideas across the company.
** Minimum Qualifications
*** Bachelor’s degree in Computer Science, Engineering, or equivalent experience.
* 5+ years of professional experience in software quality assurance, test automation, or software engineering.
* Strong programming skills in Python, with experience in test automation.
* Hands-on experience using AI-powered testing tools or AI coding assistants to improve test automation, productivity, and quality.
* Experience with CI/CD tools and processes.
* Familiarity with popular CAD environments and tooling.
* Proficient in automation and UAT test suites/frameworks.
* Experience designing QA frameworks or platforms used by multiple teams.
* Excellent problem-solving skills and attention to detail.
* Strong communication and collaboration skills.
* Understanding of software architecture and design patterns.
* Ability to work in an Agile development environment.
** Preferred Qualifications
*** Experience testing AI/ML applications or validating machine learning models.
* Experience with data validation tooling (e.g., Great Expectations) or labeling workflows.
* Familiarity with ML frameworks (e.g., PyTorch, Tensor Flow).
* Experience with data pipelines and orchestration tools (e.g., Airflow, Metaflow).
* Knowledge of MLOps practices (e.g., model monitoring, drift detection, deployment validation).
* Knowledge of ML evaluation methods, metrics, and benchmarking.
* Passion for learning new technologies and improving existing systems.
* Experience with cloud providers (e.g., AWS, Azure, Google Cloud Platform).
* Experience testing AI/ML services in production environments.
* Knowledge of experiment tracking tools (e.g., Comet, MLflow, Weights & Biases).
** The Ideal Candidate
*** You demonstrate initiative to provide solutions and to learn and develop new technologies.
* Comfortable building QA systems from scratch and writing maintainable automation.
* You enjoy learning and collaborating across global locations.
* You are comfortable working in newly forming ambiguous areas.
* You are comfortable building scalable and maintainable QA automation frameworks that will be relied on by others.
* You effectively leverage AI-powered tools to improve testing workflows and engineering productivity.
* You can communicate well…