AI​/ML Test Engineer

Modern Technology Solutions, Inc. (MTSI) is hiring an AI/ML Test Engineer in Springfield, VA. Qualified candidates must be U.S. citizens with a TS/SCI clearance and CI Polygraph
.

• Execute system-level AI/Machine Learning evaluation activities within the Test & Evaluation (T&E) environment supporting mission-focused operational testing.
• Perform integration testing, data capture, drift detection, and operational validation activities across AI/ML evaluation workflows.
• Support automated and semi-automated evaluation pipelines for AI/ML model testing and performance assessment.
• Develop, maintain, and refine data annotation workflows supporting operational evaluation requirements.
• Validate operational datasets and support testing activities across multiple models, datasets, and mission scenarios.
• Conduct integration testing of AI/ML models within operationally representative environments and workflows.
• Adapt evaluation procedures and workflows to address evolving GEOINT data conditions and operational mission requirements.
• Support high-throughput and parallel model evaluation activities to maintain alignment with mission timelines and operational priorities.
• Identify and mitigate data drift, performance degradation, and workflow issues impacting evaluation reliability.
• Collaborate with Senior AI/ML Test Engineers, data scientists, software engineers, and mission stakeholders to support scalable T&E execution.
• Contribute to technical documentation, evaluation reporting, and customer briefings supporting operational decision-making.

• 4+ years of experience supporting AI/ML testing, system integration, operational testing, or evaluation workflows.
• Familiarity with automated testing pipelines, operational data validation, and model performance assessment concepts.
• Ability to support multiple evaluation activities across dynamic operational environments.
• Experience troubleshooting technical workflow or data-related issues in support of operational objectives.
• Strong analytical, organizational, and communication skills.

• Bachelor’s degree in IT or STEM related field, or additional relevant experience in lieu of a degree.

• Experience supporting DoD, NGA, or Intelligence Community programs.
• Familiarity with AI/ML model evaluation, GEOINT workflows, and operational testing environments.
• Experience working in Agile, Dev Sec Ops , or cloud-based operational environments.
• Familiarity with annotation workflows, data capture pipelines, drift detection, and automated evaluation tools.
• Experience supporting scalable AI/ML evaluation or mission-focused T&E activities.

• Active TS/SCI clearance and the ability to pass a CI polygraph within 30 days.