Job Description
What's the opportunity?We are seeking a Lead Data and AI Quality Engineering to join our Risk and Compliance team and take ownership of quality assurance for AI and Non-Financial Risk Reporting initiatives. This role combines strategic test planning with hands-on automation and manual testing expertise to ensure the delivery of robust, high-quality solutions that meet regulatory and compliance standards.What will you do?Ensures that products being built and processes being used adhere to all standards and requirements. Applies extensive, in-depth knowledge, skills, and practices to perform complex assignments.
Test Strategy & Planning
Design and implement comprehensive test strategies aligned with risk, compliance, and regulatory requirements
Define test scope, acceptance criteria, and quality gates for AI and risk reporting solutions
Collaborate with Risk and Compliance stakeholders to understand requirements and translate them into testable specifications
Identify quality risks and develop mitigation plans in alignment with compliance frameworks
Design and code application programs for testing purposes, including test utilities and quality assurance tools
Perform comprehensive testing for developed applications, including functional, integration, and system-level testing
Validate system performance and reliability across Risk and Compliance workflows
Proven expertise building production server applications (APIs, microservices, backends)
Design and execute comprehensive test strategies for AI code agents and autonomous systems
Test AI code agents' ability to generate, review, and validate code across risk and compliance applications
Perform edge case testing and failure scenario analysis for AI agent decision-making processes
Evaluate AI agent performance metrics, including accuracy rates, false positives, and code quality standards
Validate database schemas, data models, and data integrity for risk and compliance reporting systems
Design and execute SQL queries to test data consistency, accuracy, and completeness across databases
Develop and maintain scalable, automated test frameworks and test suites using extensive Python expertise
Leverage Python libraries (pytest, requests, django, fastapi) for comprehensive test coverage
Execute automated tests as part of CI/CD pipelines and regression testing cycles
Must Haves
Extensive Python proficiency with 5+ years of hands-on development experience
Advanced Python expertise including OOP principles, design patterns, async programming, and advanced data structures
Strong software development background with experience in designing, coding, and debugging applications
Hands-on experience testing AI models and machine learning systems
Experience designing and testing AI code agents, autonomous systems, or LLM-powered applications
Demonstrated expertise in Large Language Model (LLM) testing, including prompt engineering, output validation, and hallucination detection
Experience testing Model Context Protocol (MCP) implementations and integrations
Knowledge of relational and non-relational databases (e.g., Oracle, SQL Server, PostgreSQL, MongoDB)
Solid understanding of Dev Ops practices and containerization
Proficiency with Git and Git Hub Actions for CI/CD pipeline automation and workflow orchestration
Software development certifications or advanced programming credentials
Experience with data governance, data quality frameworks, and metadata management
Experience with AI agent frameworks and orchestration tools
Knowledge of Infrastructure as Code (IaC) and containerization best practices
A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock…
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search: