AI QA Trainer - LLM Evaluation - Freelance Project

Role Overview

Join to apply for the AI QA Trainer – LLM Evaluation role at Invisible Expert Marketplace. Large‑scale language models are evolving from clever chatbots into enterprise‑grade platforms, and we need your expertise to harden model reasoning and reliability.

• Converse with the model on real‑world scenarios and evaluation prompts.
• Verify factual accuracy, logical soundness, and prompt robustness.
• Design and run test plans, regression suites, and clear rubrics with pass/fail criteria.
• Capture reproducible error traces, root‑cause hypotheses, and suggest improvements to prompt engineering, guardrails, and evaluation metrics (e.g., precision/recall, faithfulness, toxicity, latency SLOs).
• Partner on adversarial red‑teaming, automation (Python/SQL), and dashboarding to track quality deltas over time.

• Bachelor’s, master’s, or PhD in computer science, data science, computational linguistics, statistics, or a related field.
• Experience with QA for ML/AI systems, safety/red‑team, test automation frameworks (e.g., PyTest).
• Hands‑on work with LLM eval tooling (e.g., OpenAI Evals, RAG evaluators, W&B).
• Skills that stand out: evaluation rubric design, adversarial testing/red‑teaming, regression testing at scale, bias/fairness auditing, grounding verification, prompt and system‑prompt engineering, test automation (Python/SQL), high‑signal bug reporting.
• Clear, metacognitive communication – “showing your work” – is essential.

Pay range $6–$65 per hour, determined after evaluating your experience, expertise, and geographic location. As a contractor you’ll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.

Employment type:

Contract
Workplace type:
Remote
Seniority level:
Mid‑Senior Level