More jobs:
Research Scientist
Job in
Phoenix, Maricopa County, Arizona, 85001, USA
Listing for:
Anyone AI Labs
Full Time
position
Listed on 2026-11-05
Job specializations:
-
Research/Development
AI Evaluation, Research Scientist, Research Analyst
Job Description & How to Apply Below
- Conduct evaluation research to turn eval targets into original benchmark designs
- Own measurement questions including construct validity, item discrimination, and contamination
- Build evaluation packages with expert-verified ground truth and rigorous QC
- Recruit, calibrate, and review a pool of subject-matter experts across coding, agentic, and STEM fields
- Act as a technical point of contact for AI labs to translate measurement needs into evaluation designs
- Turn research findings into public benchmarks and papers for venues like NeurIPS, ICLR, and ACL
- Research background in ML evaluation or benchmarking
- Deep expertise in LLM/frontier-model benchmarking, specifically code-model and agentic evaluation
- Strong understanding of psychometrics, rubrics, and the measurement problem
- Interest in AI safety, capability elicitation, and robustness
- Ability to manage an expert pool and maintain rigorous standards
- Proficiency in the full research loop from framing to publication
- Fluency in English;
Spanish is a plus
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).