More jobs:
ML Task Auditor - Applied ML
Job in
San Francisco, San Francisco County, California, 94199, USA
Listed on 2026-09-12
Listing for:
Obsidian
Full Time
position Listed on 2026-09-12
Job specializations:
-
IT/Tech
Machine Learning/ ML Engineer, Data Scientist, AI Evaluation, AI Engineer (Applied/Software) -
Research/Development
Data Scientist, AI Evaluation
Job Description & How to Apply Below
Evaluate the quality, correctness, and methodological rigor of applied machine-learning tasks used to train and evaluate a frontier AI lab's models. You'll assess experiment design, model-selection reasoning, and evaluation methodology — and provide clear, rubric-based written feedback.
Basic Qualifications- 3+ years hands‑on applied/experimental ML (experiment design, model selection, hyperparameter tuning, evaluation methodology)
- Strong grasp of data-quality rigor: leakage detection, metric gaming, and train/test/CV hygiene
- Proficiency with standard ML frameworks (PyTorch, Tensor Flow, scikit-learn, XGBoost)
- Ability to critique ML claims against evidence and reproduce results
- Competition / benchmark experience (e.g., Kaggle)
- Graduate research or publication record in applied ML
- Prior task-grading or peer-review experience
Note:
this role evaluates applied/experimental ML rigor — it is not an LLM-application-building or MLOps role.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×