Senior QE Automation Engineer; AI Solutions

Role Name:
Senior QE Automation Engineer (AI Solutions)

Location:

Toronto / 4 days a week on-site,
F2F interview required

Position Overview

We are seeking an experienced Senior QE Automation Engineer to join our AI Development Lifecycle AIDLC team This role focuses on ensuring quality and reliability of AIML systems through comprehensive test automation strategies specialized AI model validation and robust quality engineering practices

Develop and maintain automated testing frameworks for AIML applications and pipelines

Create comprehensive test automation strategies covering unit integration system and endtoend testing

Implement continuous testing practices within CICD pipelines for AI model deployment

Develop automated tests for model training inference and monitoring systems

Validate AI model performance accuracy bias fairness and robustness

Design test cases for model drift detection and data quality validation

Implement automated testing for model versioning and AB testing scenarios

Conduct performance and load testing for ML inference endpoints

Validate data pipelines feature engineering and ETL processes

Build and maintain test infrastructure for AI workloads

Integrate testing tools with MLOps platforms

Implement monitoring and observability for test automation systems

Manage test data and synthetic data generation for AI testing

5 years of experience in QA automation engineering

2 years of handson experience testing AIML systems or data intensive applications

Expertise in test automation frameworks Pytest Selenium Playwright Cypress or similar

Experience with API testing tools Postman REST Assured or similar

Proficiency with CICD tools Jenkins Git Lab CI Git Hub Actions

Strong understanding of ML concepts model training evaluation metrics inference feature engineering

Experience testing machine learning models classification regression NLP

Knowledge of model evaluation metrics accuracy precision recall etc

Understanding of data quality testing and validation techniques

Familiarity with ML frameworks

Experience with model monitoring and observability tools

Experience with cloud platforms AWS Lambda Azure ML

Knowledge of containerization Docker Kubernetes

Understanding of distributed systems and microservices architecture

Experience with version control systems Git and collaborative development workflows