Expert Talent Pool, Evals and Benchmarks; Associate
Job in
Philadelphia, Philadelphia County, Pennsylvania, 19113, USA
Listing for:
Clera
Full Time
position
Listed on 2026-11-17
Job specializations:
-
IT/Tech
-
Research/Development
Job Description & How to Apply Below
Responsibilities
- Write questions and tasks that isolate and test specific model capabilities
- Develop rubrics defining strong, partial, and poor answers
- Create reference answers and edge cases to challenge AI models
- Grade model outputs against rubrics and explain scoring decisions
- Analyze failed answers, group error patterns, and document findings
- Compare multiple models using identical prompts
- Validate benchmarks and flag compromised or ineffective questions
- Participate in small user studies evaluating model outputs
- Zero to five years of professional experience
- QA, test engineering, assessment design, or ML evaluation experience
- Experience writing test cases, exam questions, or rubrics
- Comfort using spreadsheets or Python to analyze results
- Clear written communication
- Experience with evaluation tools such as Inspect, lm-eval-harness, HELM, or promptfoo
- Compensation of $30 to $60 per hour, depending on project, field, and experience
- Payment for hours worked
- Fully remote; projects typically last one week to several months
Position Requirements
10+ Years work experienceTo View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction