Queue Lead - AR

• Own work queues: intake, triage, prioritization, task assignment, and SLA delivery
• Monitor task queue and backlog control
• Identify and troubleshoot bottlenecks
• Monitor quality at project and individual annotator level
• Perform QA tasks such as making corrections and providing feedback
• Manage annotator performance by unassigning underperformers and retaining high-performers
• Build and maintain dashboards to track throughput, disagreement rates, and defect rates
• Oversee Slack channels to provide support and answer participant questions
• Train new annotators on project guidelines
• Conduct webinars and host office hours for participants

• Proven, direct experience running queues and quality on AI data programs
• 3+ years in high-throughput operations (data labeling, trust & safety, BPO, content ops, or CX ops)
• Ability to identify low-effort or AI-generated responses
• Proven ability to design and run calibration processes from scratch
• Proficiency in spreadsheets (SQL, BI, Looker, or Tableau is a strong plus)
• Experience managing contributor performance at scale
• Comfortable with ambiguity and fast-changing priorities

• Experience with ML data or LLM evaluation workflows
• Distributed team coordination across time zones
• Consulting-style experience advising clients on AI training methodology

Prolific is building the biggest pool of quality human data in the world, providing the human data infrastructure necessary for developing accurate and aligned AI systems.