Analyst, AI QA Engineer

Analyst, AI QA Engineer

Why We Work at Dun & Bradstreet

We are at a transformational moment in our company journey - and we’re so excited about it. Each day, we are finding new ways to strengthen our award-winning culture, and to accelerate creativity, innovation and growth. Our purpose is to help customers improve business performance with Dun & Bradstreet’s Data Cloud and Live Business Identity, and we’re wildly passionate and committed to this purpose.

So, if you’re looking to make an immediate impact at a company that welcomes bold and diverse thinking, come join us!

The Role:

Dun & Bradstreet Technology and Corporate Services India LLP is the global center of excellence for providing technology and analytics solutions and services to Dun & Bradstreet and its clients globally. We specialize in delivering predictive models and decision management solutions to convert raw data into actionable insights. Our customers include banks, financial services organizations, credit bureaus, rating agencies and thousands of businesses across the globe.

Headquartered in Hyderabad (India), the company employs high caliber data scientists, economists, analytics professionals and data engineers from some of the finest institutions in the country striving to create more transparent credit & business environment and robust economies.

We are looking for a highly skilled AI Tool / Agent Testing Engineer to evaluate, validate, and ensure the reliability of AI agents, AI automation tools, and agentic workflows used across our analytics platform. This role blends test engineering, GenAI understanding, Python programming, and agent development lifecycle knowledge. You will work closely with Data Science, AI Engineering, and Platform teams to ensure that AI agents behave predictably, safely, and in alignment with business and compliance requirements.

Key Responsibilities:

• Design, develop, and execute test plans and test cases to validate the functionality, performance, and reliability of AI/ML systems.

• Collaborate with the development team to understand model requirements and identify potential areas of weakness.

• Perform data validation to ensure the quality, integrity, and representativeness of the datasets used for training and testing.

• Test for and identify potential biases in AI models to ensure fairness and ethical compliance.

• Analyze and report on model performance using key metrics (e.g., accuracy, precision, recall, F1-score).

• Ensure accuracy, consistency, tool-call reliability, trace quality, and guardrail adherence.

• Assess regression, functionality, performance, safety, and hallucination risks.

• Document and track defects and work with developers to facilitate their resolution.

• Assist in the development and maintenance of automated testing frameworks for AI applications.

• Conduct exploratory testing to discover edge cases and unexpected behaviors in our AI systems.

• Stay current with the latest advancements in AI testing methodologies and tools.

• Produce test plans, scenario libraries, coverage reports, and defect logs.

• Deliver insights to Data Science & Engineering teams to improve reliability.

Key Requirements:

• Bachelor's degree in Computer Science, Engineering, Statistics, or a related technical field.

• Solid understanding of software testing principles and the software development lifecycle (SDLC).

• Basic programming proficiency, preferably in Python, for writing test scripts and analyzing data.

• A foundational understanding of machine learning concepts (e.g., supervised/unsupervised learning, classification, regression).

• Strong analytical and problem-solving skills with an exceptional eye for detail.

• Excellent communication and collaboration skills, with the ability to articulate technical issues clearly.

• Prior internship or project experience in software testing or a data-related field.

• Familiarity with machine learning libraries and frameworks such as Tensor Flow, PyTorch, or Scikit-learn.

• Experience with testing tools and frameworks like PyTest, Selenium, or Postman.

• Knowledge of SQL for querying and validating data.

• Familiarity with version control systems like Git.

• A genuine…