Operations Specialist

Operations Specialist

The future of AI whether in training or evaluation, classical ML or agentic workflows starts with high-quality data.

At Human Signal, we're building the platform that powers the creation, curation, and evaluation of that data. From fine-tuning foundation models to validating agent behaviors in production, our tools are used by leading AI teams to ensure models are grounded in real-world signal, not noise.

Our open-source product, Label Studio, has become the de facto standard for labeling and evaluating data across modalities from text and images to time series and agents-in-environments. With over 250,000 users and hundreds of millions of labeled samples, it's the most widely adopted OSS solution for teams working on building AI systems.

Label Studio Enterprise builds on that traction with the security, collaboration, and scalability features needed to support mission-critical AI pipelines powering everything from model training datasets to eval test sets to continuous feedback loops. We started before foundation models were mainstream, and we're doubling down now that AI is eating the world. If you're excited to help leading AI teams build smarter, more accurate systems we'd love to talk.

Level: Individual Contributor Compensation: $55,000 $ 75,000

Location:

San Francisco, CA

Human Signal specializes in operationally complex, multimodal data collection and annotation delivering the datasets that frontier AI research requires and remote workforce marketplaces can't. We own projects end-to-end, from scoping and protocol design through final delivery, running on-site and distributed expert work forces across 50+ knowledge domains, 30+ languages, and 75+ countries. Our work spans RLHF, evals, red-teaming, and custom multimodal data creation, all powered by Label Studio Enterprise and built on a foundation of rigorous quality workflows, ethical sourcing, and full data security.

This role sits at the operational core of that delivery engine responsible for ensuring our clients get the highest-quality data on time, every time.

This role is not for everyone. Human Signal Services operates at the intersection of frontier AI research and large-scale human data delivery and the work is fast, demanding, and unforgiving of dropped balls. You'll own complex, high-stakes data programs end-to-end, managing expert work forces, navigating shifting customer requirements, and holding quality and delivery timelines simultaneously. There is no playbook handed to you.

You will build it, break it, and rebuild it better. If you thrive under pressure, take personal pride in operational excellence, and don't quit when a project gets hard this is the role for you

The day-to-day is intense by design. You'll juggle multiple programs at once, each with its own contributors, quality standards, and customer expectations. You'll make hard calls with incomplete information, hold contributors and vendors accountable, and find creative solutions to problems that don't have obvious answers. The expectation is simple: own the outcome, no excuses. High performers here grow faster than anywhere else in the industry.

The bar is high and that's the point.

• Build and drive critical operational processes to meet day-to-day customer delivery commitments
• Manage daily operations across multiple Fellow or contributor programs, including onboarding, workflow coordination, and ongoing program health
• Maintain operational trackers and documentation to ensure consistent execution and follow-through across the team
• Monitor contributor engagement and flag retention or performance risks before they become delivery issues
• Create an effective feedback loop between frontline operations, product, strategy, and customers
• Identify recurring bottlenecks and recommend process improvements; run pilots and experiments to test solutions
• Collaborate with SPLs, TPMs, Customer Operations, and Product teams to improve processes for new and existing customers

• 2+ years of experience in an operational role or top-tier consulting firm
• Undergraduate degree in an analytics-heavy field (Engineering, Economics, or related)
• Action-oriented; balances creative problem-solving with scrappy execution
• Strong analytical and process-improvement capability
• Proficiency in SQL or demonstrated analytical tooling skills

• Experience in marketplace, logistics, or workforce platform operations
• Familiarity with tools like Zendesk, Notion, or Airtable
• Exposure to AI/ML workflows, data annotation, or contributor management
• Entrepreneurial mindset; comfortable in fast-moving, ambiguous environments