Data Annotation & AI QA Specialist
Job in
San Antonio, Bexar County, Texas, 78201, USA
Listing for:
Longwood University
Part Time
position
Listed on 2026-11-16
Job specializations:
-
IT/Tech
Job Description & How to Apply Below
As a Data Annotation & AI QA Specialist
, you will help build and maintain our "golden benchmark dataset"—the foundational ground-truth dataset used to evaluate AI-generated project briefs, listing claims, and expert proof submissions across our platform.
This role is ideal for students or recent graduates interested in gaining direct experience in AI Quality Assurance, Natural Language Processing (NLP), and Data Operations
. You will work at the intersection of human evaluation and machine learning model validation.
- Benchmark Dataset Creation: Annotate, clean, and structure text and multi-modal data pairing complex business requirements with 9-line JSON project briefs.
- Claim & Proof Verification: Audit AI-generated listing claims against raw technical proof records (code snippets, API logs, and execution outputs).
- Quality Assurance & Guidelines: Help refine annotation guidelines and schema rules to ensure high inter-annotator agreement (>95%).
- Model Output Evaluation: Perform systematic human-in-the-loop (HITL) quality audits on outputs generated by fine-tuned LLMs and automated tools.
- Edge Case Flagging: Identify ambiguous, misleading, or hallucinated AI outputs and feed corrected samples back into our training pipeline.
- Educational Background: Currently pursuing or recently completed a Bachelor’s or Master’s degree in Computer Science, Data Science, Information Systems, Linguistics, or a related quantitative field.
- Attention to Detail: Meticulous approach to data precision—you take pride in identifying subtle errors or inconsistencies in data.
- Technical Literacy: Basic familiarity with Python, JSON structures, regular expressions, or spreadsheet/data management tools (Pandas, Excel, Airtable).
- Analytical Thinking: Ability to interpret technical documentation, system logs, and business workflows quickly.
- Communication: Strong written English skills for writing clear annotation instructions and feedback reports.
- Understanding of basic machine learning concepts (precision, recall, ground truth, fine-tuning).
- Hands-on AI
Experience:
Work directly with state-of-the-art LLM pipelines and real-world training/evaluation datasets. - Flexible Remote Environment: Work asynchronously from anywhere with structured milestone goals.
- Mentorship & Mentoring: Direct collaboration with senior machine learning and automation engineers.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction