Automation Tester; Playwright

Position Title:

Automation Tester (with Playwright)

Start Date:

ASAP Approved On‑Site Location(s):
Irving, TX On‑Site

Schedule:

Hybrid (3 days/week in office) # Of Openings: 1 Skillset: API, UI, Performance Testing, Playwright Overview:

The IAM Tools Engineering team (ITE) is looking for a highly motivated and skilled AI Test Engineer to join our growing team. The AI Test Engineer will be responsible for designing, developing, and executing comprehensive test strategies and plans for AI/ML models and applications. This role requires a strong understanding of AI principles, machine learning workflows, data quality, and various testing methodologies specific to intelligent systems.

The ideal candidate will work closely with the ITE engineering team and application users, have related experience developing tests in an iterative manner, be passionate about quality, possess excellent problem‑solving skills, and thrive in a collaborative, fast‑paced environment. This person must have a proven track record of demonstrating their ability to effectively design and develop test frameworks, execute tests manually and in an automated manner, and have a strong technical focus.

Critical success factors will include the ability to effectively engage in a matrixed organization, develop partnerships with many business, analyst, and development teams, and drive the quality assurance strategy for a cutting‑edge AI enable self‑service identity solution.

• Develop and implement robust test plans and strategies tailored for AI/ML systems, including defining test objectives, methodologies, and scenarios.
• Review design specifications to understand the scope, requirements, and requested functions of the software product, partnering with the team during delivery of the stories to ensure stories are accurately implemented.
• Identify appropriate parameters, functions, and data to test and validate requirements and acceptance criteria; ensuring a successful development and deployment that meets the business needs.
• Ability to create effective granular and end‑to‑end test scenarios for user interface, data transactions, integrated systems, and functional aspects of web‑based software applications.
• Collaborate with engineers, and product managers to understand AI model requirements, use cases, and potential risks.
• Define quality metrics for AI models, focusing on aspects like accuracy, fairness, robustness, and interpretability.
• Design and create detailed test cases for various types of AI testing, including validating AI model outputs against expected behavior, prompt‑response accuracy, and contextual understanding.
• Assessing model speed, latency, throughput, and scalability under different load conditions.
• Bias and Fairness Testing:
  Identifying and mitigating biases in training data and model predictions to ensure ethical and equitable AI behavior.
• Evaluating model resilience against various inputs, including malicious or unexpected data.
• Data Quality Testing:
  Ensuring the integrity, consistency, relevance, and representativeness of data used for training and testing AI models.
• Identifying vulnerabilities in AI systems and ensuring compliance with data protection standards (e.g., GDPR, CCPA).
• Verifying seamless integration of AI models with other systems, APIs, and existing software.
• Execute manual and automated test cases, analyzing results and identifying defects, anomalies, and inconsistencies.
• Implement and maintain automated testing frameworks and tools specifically designed for AI/ML models and pipelines.
• Develop test scripts using programming languages commonly used in AI/ML (e.g., Python, Java, JavaScript).
• Utilize AI‑specific testing tools and frameworks to enhance testing efficiency and accuracy.
• Integrate testing processes into Continuous Integration/Continuous Deployment (CI/CD) pipelines.
• Accurately document, track, and prioritize defects and issues found during testing.
• Work closely with development teams to ensure timely resolution of identified problems.
• Prepare comprehensive test reports, providing actionable insights into AI model performance and quality.
• Work with leaders, stakeholders, SMEs, scrum team,…