More jobs:
Job Description & How to Apply Below
Overview
We are building and evaluating frontier models and need experienced machine learning and natural language processing practitioners to act as ground-truth experts. You will author challenging, real-world tasks, generate reference solutions, and evaluate model outputs to surface reasoning and capability gaps in target models.
The work centers on designing robust ML and NLP problems, building them out with executable tests where applicable, and analyzing how models and agents behave against them. All applicants are expected to have working proficiency in Python.
This is a part-time, fully remote role within the United States, at approximately 20 hours per week.
Key Responsibilities
Task design and development:
Design challenging, real-world ML and NLP problems drawn from your area of expertise (e.g., model training and evaluation, language understanding and generation, retrieval, applied ML pipelines) that target specific capability gaps in frontier AI models.
Spec and golden-solution generation:
Integrate problems into an agentic development environment, preparing all necessary components using Python.
Evaluation and analysis:
Evaluate target model performance on your tasks.
Headroom identification:
Identify tasks where the target model fails, and classify the nature of the failure.
Collaborate with other experts:
Work alongside fellow subject-matter experts to keep evaluations consistent and accurate.
Qualifications
Deep, hands-on experience in machine learning and/or natural language processing, from applied industry work, research, or a graduate/PhD background in the field.
Working proficiency in Python, applied in research, industry, or open-source work (not theoretical familiarity).
Strong command of modern ML/NLP methods, including model training and evaluation, transformers and large language models, and standard tooling.
Ability to engage reliably for approximately 20 hours per week.
Past experience in AI training, model evaluation, or data annotation is preferred.
Strong written communication and the ability to work independently and manage your own time.
Pay rate: $80–110/hr · ~20 hrs/week
Note that applications are not being accepted from your jurisdiction for this job currently via this jobsite. Candidate preferences are the decision of the Employer or Recruiting Agent, and are controlled by them alone.
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
Search for further Jobs Here:
×