AI Validation SME

AI Validation SME

Join Sky Solutions, LLC - Pioneering Digital Transformation Excellence!

Sky Solutions is a trusted partner in government contracting, empowering federal agencies with cutting-edge IT solutions. Our expertise spans AI-first enablement, delivering innovative solutions and advanced cybersecurity, fortifying digital ecosystems against emerging threats and digital transformation through low-code/no-code development, cloud solutions, and agile methodologies. We simplify complex processes and deliver breakthrough results for federal clients across healthcare, financial services, and national security.

Location:

Tysons Corner, VA (Hybrid / Remote) Duration:
Fulltime

Note:

Active MBI clearance is required for this role.

Job Summary

Serve as an AI Validation Subject Matter Expert (SME) responsible for validating AI, Generative AI, and AI-enabled solutions to ensure accuracy, reliability, performance, security, explainability, robustness, and compliance with defined business and technical requirements. Develop and execute validation strategies, test plans, evaluation criteria, and validation procedures throughout the AI solution lifecycle, including appropriate human-in-the-loop validation for critical AI-generated outputs and decisions.

Key Responsibilities

• Develop AI validation strategies, test plans, methodologies, and acceptance criteria.
• Validate AI models and applications for accuracy, consistency, reliability, performance, bias, explainability, and robustness.
• Design and execute functional, integration, regression, performance, and AI-specific validation testing.
• Evaluate AI-generated outputs against business requirements, expected outcomes, and defined quality thresholds.
• Validate data quality, training/validation datasets, prompts, model responses, and AI decisioning logic.
• Incorporate human-in-the-loop validation processes to ensure critical AI decisions and outputs are reviewed and approved by appropriate subject matter experts.
• Identify, document, and track AI defects, anomalies, risks, and performance issues through resolution.
• Validate Pega AI, Generative AI, intelligent automation, and AI-enabled decisioning/workflows, where applicable.
• Support testing and validation of AI integrations with enterprise applications, APIs, databases, and other systems.
• Assess AI solutions for security, privacy, compliance, transparency, and Responsible AI requirements in coordination with the AI Governance SME.
• Establish monitoring and ongoing validation approaches for AI solutions after production deployment.
• Collaborate with AI engineers, data scientists, architects, developers, business stakeholders, cybersecurity, and governance teams.
• Prepare validation reports, test results, risk findings, and executive-level summaries.

Required Qualifications

• Bachelor's degree in computer science, Information Technology, Data Science, Engineering, Mathematics, or a related field.
• 8+ years of experience in software testing, quality assurance, data validation, AI/ML validation, technology risk, or a related discipline.
• Demonstrated experience developing and executing test strategies, validation plans, test cases, and acceptance criteria.
• Experience validating AI, Machine Learning, Generative AI, or AI-enabled applications.
• Knowledge of AI evaluation concepts including accuracy, reliability, robustness, bias, explainability, performance, and data quality.
• Experience with functional, integration, regression, performance, and data validation testing.
• Experience evaluating AI-generated outputs, model responses, prompts, decisioning logic, or AI-enabled workflows.
• Experience implementing or supporting human-in-the-loop validation and review processes for critical AI outputs or decisions.
• Knowledge of AI security, privacy, compliance, Responsible AI, and risk management principles.
• Strong analytical, technical documentation, problem-solving, and stakeholder communication skills.
• Experience working with cross-functional AI, data, architecture, cybersecurity, development, and business teams.
• Experience with Pega AI, Generative AI, intelligent automation, or AI-enabled decisioning is preferred.
• Federal Government or highly regulated environment experience is preferred.

Compensation

The expected compensation range for this position is $140,000 - $200,000 annually. Compensation is determined in good faith and may vary based on job-related factors, including relevant experience, skills, education, certifications, clearance level, labor category alignment, contract requirements, work location, internal equity, and business needs. The posted range reflects the Sky's reasonable estimate at the time of posting and does not guarantee a specific salary or rate.

Final compensation will be determined based on the selected candidate's qualifications and applicable contractual and business considerations.

Sky Solutions Benefits & Culture at a Glance

• Healthcare & Wellness
  • Medical, dental, and vision coverage tailored to your needs.
  • Supplemental options like life insurance,…