Sr. Gen AI - Tester

Title:
Sr. Generative AI Tester (Airline Domain) Exp.: 8-9+ Yrs. Key

Skills:

Python | Generative AI | LLMs | AI Testing | LLM Evaluation | AI Agent Testing | Prompt Testing | Hallucination Testing | AI Agents | Agentic AI | Airline Domain | QA Automation | Responsible AI.

We are seeking an experienced AI Engineer with strong expertise in Generative AI, LLMs, AI Agents, and AI Testing
. The primary focus of this role is to ensure the quality, reliability, accuracy, security, and performance of AI-powered solutions used within the airline industry. The ideal candidate will have hands-on experience developing AI applications and establishing robust testing frameworks for LLMs, AI agents, and conversational AI systems.

• Develop and support AI applications using Python, Generative AI, and Large Language Models (LLMs).
• Design, execute, and automate comprehensive AI test strategies across the AI application lifecycle.
• Validate AI models, LLMs, AI agents, and agentic workflows for accuracy, consistency, reliability, and safety.
• Perform functional, integration, regression, performance, security, and user acceptance testing for AI solutions.
• Conduct LLM evaluation, prompt testing, hallucination testing, bias and fairness assessments, and guardrail validation
  .
• Develop automated frameworks for testing AI agents, chatbots, copilots, and autonomous workflows.
• Define and track AI quality metrics, benchmarks, and acceptance criteria.
• Identify, analyze, and resolve defects related to AI outputs, workflows, and model behavior.
• Collaborate with business stakeholders and development teams to ensure production-ready AI solutions for airline operations.

• Strong programming expertise in Python
  .
• Hands-on experience with LLMs, Generative AI, RAG architectures, Vector Databases, and AI Models
  .
• Experience with AI frameworks such as Lang Chain, Lang Graph, CrewAI, Auto Gen, Llama Index, or Semantic Kernel
  .
• Experience developing and testing AI Agents, Agentic AI solutions, Chatbots, and Conversational AI platforms
  .
• Strong expertise in AI Testing
  , including:
• LLM Evaluation & Validation
• AI Agent Testing
• Hallucination Testing
• Bias & Fairness Testing
• Functional Testing
• Integration Testing
• Regression Testing
• Performance & Load Testing
• Security & Responsible AI Testing
• Test Automation for AI Applications
• Experience creating AI quality frameworks, test cases, evaluation datasets, and benchmarking methodologies.
• Familiarity with Azure OpenAI, AWS AI, Google AI services, or equivalent platforms.

• Experience in the Airline, Aviation, Travel, or Transportation domain is required.
• Knowledge of airline operations, reservations, customer service, crew management, airport operations, or passenger experience solutions is an advantage.

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Data Science, Engineering, or related field.
• Certifications in AI, Machine Learning, Software Testing, or Cloud Technologies are preferred.