Evaluation and Research Director, USAFA

Evaluation and Research Director, USAFA - (2547)

Yorktown Systems Group, Inc. (YSG) is seeking a highly qualified Evaluation and Research Director to provide comprehensive Warfighter Training Support Services at the United States Air Force Academy (USAFA). The candidate will support the planning, development, execution, management, and evaluation of military training for approximately 4,200 cadets annually during both the academic year and summer training programs.

• Serve as the lead and primary Evaluation and Research representative, providing management and oversight of the Evaluation & Research Team.
• Convert execution findings into future planning inputs.
• Aggregate and analyze performance data across events, cohorts, and training cycles within Moodle/TUTOR VSP to identify trends, systemic strengths, and recurring performance gaps.
• Work with USAFA leadership, instructional designers, and training support personnel to ensure evaluation methodologies, performance measures, and assessment requirements remain aligned with program objectives.
• Convert evaluation findings, performance trends, and lessons learned into recommendations that support refinement of warfighter competencies, training effectiveness, learning outcomes, and program improvement initiatives.
• Convert execution observations, metrics, and lessons learned into resource planning inputs that inform future assessments, refinement, and recommendations.
• Capture observations, rubric results, AAR inputs, instructor feedback, participant performance data, and lessons learned during and after training events.
• Collectively execute all program evaluation activities across the WTIS framework.
• Apply Kirkpatrick’s four-level training evaluation model within a repeatable process that integrates assessment tool development, disciplined data capture, structured AAR consolidation, and performance analysis within the required environment and tools.
• Define and maintain the data structure and workflows required to capture, store, and export evaluation data in a manner that supports consistent analysis.
• Conduct analytical review of existing and available Program of Instruction (POI), courseware, lesson materials, exercise scenarios, training schedules, TLOs, ELOs, primary tasks and objectives, evaluation criteria, recurring training events, historical planning products, and after‑action inputs.
• Use assessment data and other evaluation inputs to develop dashboard reporting, leadership briefings, annual program assessments, and evidence‑based recommendations for program improvement.
• Design and execute qualitative and quantitative research studies that assess cadet development, training effectiveness, and progression across the military training continuum.
• Prepare executive‑level reports, dashboards, briefings, and recommendations for USAFA leadership utilizing evaluation findings and performance trends.
• Ensure evaluation methodologies, data collection activities, and reporting processes meet accepted standards for program evaluation, research integrity, and statistical validity.

• 8+ years of experience as an evaluation and/or research lead.
• Graduate degree in Program Evaluation, Education Statistics, Social Science, Research Methods, Organizational Development, or a related field.
• Experience supporting assessment frameworks, AAR methodology, rubric development, data analysis, trend reporting, and presentation of findings to stakeholders.
• Experience evaluating programs/projects; creation of evaluation plans and logic models; developing data collection tools and procedures; facilitating data collection; conducting statistical analysis of data; writing evaluation reports and providing statistical analysis of programs of similar scale and duration.
• In‑depth understanding of program evaluation standards (propriety, utility, feasibility, and accuracy).
• Ability to evaluate the implementation and effectiveness of key initiatives and programs through quantitative and qualitative research methods, providing results that inform decision‑making.
• Ability to capture, conduct analysis, and organize collected data.
• In‑depth understanding of research…