Evaluation Lead, AI Operations

About The Role

At Fetch, we’re building AI and automation systems that make our work smarter, faster, and more scalable. The AI Operations team ensures our models, automations, and LLM systems perform with quality, reliability, and measurable impact. As an Automation Lead, you’ll translate business and functional goals into scalable systems, define quality measurement, and make automation and evaluation durable, high‑impact capabilities across the organization.

This full‑time role can be based in one of our U.S. offices or worked remotely from anywhere in the United States.

• Lead complex, high‑impact automation and evaluation initiatives across workflows or teams
• Architect end‑to‑end scalable workflows that integrate datasets, evaluations, automations, and HITL processes
• Define best practices for datasets, evaluation methodologies, and quality measurement
• Influence priorities and partner with Engineering, Product, and AI teams to deliver solutions
• Redesign workflows, tooling, and processes to improve performance and durability at scale
• Define success metrics, communicate impact, and recommend actions to leadership
• Mentor teammates and elevate the team’s automation and evaluation capabilities
• Stay current on emerging tools and approaches, pilot and translate them into actionable improvements for the team

• 5+ years of experience leading complex automation or evaluation initiatives across systems or teams
• Experience designing, building, and scaling evaluation frameworks, datasets, and quality systems for LLMs or AI products
• Strong understanding of AI/LLM workflows, automation platforms, and system design
• Experience influencing cross‑functional stakeholders and aligning priorities
• Demonstrated ability to define metrics and drive measurable business impact

• Familiarity with data pipelines, APIs, or production systems
• Experience mentoring or leading technical contributors
• Background in AI Operations, applied AI, or process optimization

We offer competitive compensation packages including base salary, equity, and benefits. The base salary range for this position is $97,000 - $114,000. Discover our benefits and how our employees live rewarded at

• Equity in Fetch for full‑time employees
• 401(k) match up to 4%
• Comprehensive medical, dental, and vision plans for employees and their pets
• Annual educational reimbursement of up to $10,000
• Employee Resource Groups and inclusion initiatives supported by the Inclusion Council
• Paid time off with 9 paid holidays and a year‑end week‑long break
• Robust leave policies: 20 weeks paid parental leave for primary caregivers, 14 weeks for secondary caregivers, and flexible return‑to‑work schedule
• Childcare assistance: one‑time $2,000 incentive to cover childcare, clothing, diapers, and more
• Flexible work environment with fully remote options and hardware/software support

Fetch is an equal‑opportunity employer that embraces diversity, inclusion, and respect for all individuals. We do not discriminate on the basis of race, color, religion, gender, gender identity or expression, sexual orientation, age, national origin, marital status, veteran status, disability, or any other characteristic protected by applicable law. We provide reasonable accommodations to qualified individuals with disabilities or those with sincerely held religious beliefs, as required by law.

If you need assistance with the application process or require an accommodation, please contact us at