Data Science Methodology SME
Jacksonville, Duval County, Florida, 32099, USA
Listed on 2026-11-01
-
IT/Tech
AI Evaluation
Evaluate large language model behavior in complex data pipelines, write structured test cases, and improve analytic methods as a remote contractor earning $60 to $100 per hour.
Remote Hourly · $60–$100/hr
$60–$100/hr
Compensation
Contract, part-time
Engagement
Remote
Location
Oct 23, 2026
Posted
Open in 52 countries
India + more
- Algeria
- Argentina
- Austria
- Bahrain
- Bangladesh
- Belgium
- Brazil
- Canada
- Chile
- Colombia
- Denmark
- Egypt
- France
- Germany
- Ghana
- Greece
- India
- Indonesia
- Italy
- Japan
- Jordan
- Kazakhstan
- Kenya
- Kuwait
- Kyrgyzstan
- Lebanon
- Malaysia
- Mexico
- Morocco
- Netherlands
- Nigeria
- Oman
- Pakistan
- Palestine
- Peru
- Philippines
- Portugal
- Qatar
- Saudi Arabia
- Singapore
- South Africa
- Spain
- Sri Lanka
- Switzerland
- Taiwan
- Thailand
- Tunisia
- United Arab Emirates
- United Kingdom
- United States
- Uzbekistan
- Vietnam
You will support AI training projects that test how well large language models handle real-world data science and analytic workflows. Your work will cover scaled analysis, pipeline integration, automation failures, over-refusal, and the practical value of model outputs.
You will turn analytic scenarios into structured evaluation cases, assess model responses, and explain where reasoning, source handling, limitations, or methodology need improvement.
- Map scaled-analysis and pipeline-integration scenarios to an established evaluation taxonomy.
- Write detailed cases covering automated triage, data enrichment, analysis at scale, and realistic failure modes.
- Evaluate and grade model responses for source handling, reasoning, limitations, and practical usefulness.
- Identify over-refusal that could block legitimate analysis or interrupt pipeline continuity.
- Document difficult edge cases and emerging concerns in analytic methodology.
- Give clear written and verbal recommendations to support iterative improvement.
This is a remote, part-time contractor role for candidates in the countries specified for the project. The work requires strong written English and the ability to explain complex technical ideas clearly.
- Pay: $60 to $100 per hour.
- Location:
Open to candidates in the listed countries. - Experience level:
Entry level. - Experience integrating large language models, including government GPT variants or equivalent systems, into complex data or analytic pipelines.
- Experience conducting analysis at scale, rather than only one-off analysis.
- Understanding of automation failure modes, evaluation metrics, and analytic methodology.
- Experience teaching, mentoring, or providing technical leadership in data science or a related field.
- Familiarity with evaluation frameworks or benchmark design.
- Helpful background:
Defense, intelligence, or government analytics experience, plus the ability to work in distributed international environments with minimal clearance constraints.
AI training is the human work behind modern artificial intelligence, including testing model answers, rating performance, and preparing examples that help systems improve. Open Train helps people find these projects, build a profile, and develop a portfolio of AI training and data-labeling experience.
Requirements- Experience:
Entry level
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).