Research Psychometrician

Flash Academy is hiring a Research Psychometrician on a fixed term contract to do the measurement and statistical work on a twelve month, government funded research project in schools. You will audit our historical data, fix each analysis plan before the results are seen and run the final evaluation.

Flash Academy is a UK learning and assessment platform for people learning English as an additional language, used by more than 700,000 learners in over 2,400 schools, colleges and workplaces. The project tests whether schools can identify special educational needs earlier and more fairly among pupils who are still learning English. It is research, not product development: the results are reported whatever they show, and nothing is built into the product during the project.

Contract Fixed term to 30 September 2027, four days a week

Start December 2026, or January 2027 at the latest. The first data audit is covered by the existing team until you start.

Reports to Chief Technology Officer

Location Hybrid or remote within the UK. No school visits

Data You work with pseudonymised pupil records in a separate research environment, never with the live product

As our Research Psychometrician, you will do the project’s analysis through four phases, about 178 working days between your start and September 2027. Every analysis plan is fixed and lodged with the funder before the results are seen and an independent measurement specialist reviews and signs your plans, threshold and evaluation.

Phase
When
What you will do
What exists at the end

1. Audit and plan To December 2026 Audit the historical pupil records: how complete each field is and whether it means the same thing across the years covered. Count the usable sample, including pupils with two assessment sittings and two terms of app use. Write the analysis plan for historical records and lodge it before any analysis. Build the first comparison profiles of typical progress.

Design the scoring for the new pupil tasks. Data audit report. Exploratory analysis plan lodged. First comparison profiles and a progress note.

2. Scoring fixed January and February 2027 Update the comparison profiles. Fix the scoring rules and the rules for choosing which pupils sit the tasks. Build the pipeline for incoming task data. Revise scoring after the speech and language therapist’s fairness review. Scoring and selection rules fixed under version control.

3. Modelling and evaluation March to July 2027 Report whether platform data improves on a simple comparison based on starting level and time in schooling. Write the analysis plan for the study in schools and lodge it before any model is built. Name the schools reserved for the final evaluation. Set the threshold at which a pupil is raised for professional review.

Measure agreement between specialist reviewers. Run the single final evaluation on the reserved schools. Early findings for the second decision point. Confirmatory analysis plan lodged. Threshold record. Evaluation report covering identification rate, dependability of confidence and review burden.

4. Fairness and reporting August and September 2027 Analyse performance by home language, year group and starting level, and say where numbers are too small to conclude. Supply figures for the usability and commercial reports. Write the methods, findings and limitations of the final report. Document data and code for the archive. Fairness analysis. Final report sections. Archive documentation.

The study plans 400 to 800 pupils in at least eight schools, 100 to 150 specialist case file reviews and about 60 face-to-face assessments.

The project needs a different model for each kind…