Senior Software Engineer — AI​/ML

As a Software Engineer on the Evaluation Engineering team, you'll build systems to power large-scale AI workloads for top tier AI research labs. You’ll work closely with other engineers, product managers, and field team members to ensure that Snorkel’s frontier AI datasets meet and surpass the capabilities of the most advanced foundation models. You will launch agents into production to accomplish long running reasoning and verification tasks.

The Evaluation Engineering team owns critical systems for building high quality post-training and benchmark datasets, integrating with the latest foundation model technology to push the frontier of models used globally across coding, mathematics, law, medicine, and other advanced domains.

• Own the architecture, design, development, and operations of large-scale systems designed for AI/ML tasks including distributed compute systems, data management systems, data engineering workflow systems, and end user experiences
• Recognize and act on opportunities to integrate the latest agentic and foundation model technologies to power eval workflows
• Prototype, optimize, and maintain scalable back-end services that will power new foundation model development tools
• Design extensible and testable interfaces between internal services including the underlying storage and data models
• Be an engaged team player in a customer-focused cross-functional environment where you will feel excited to take on whatever is most impactful for the company and product
• Work a hybrid schedule with 3 days per week in one of our offices in San Francisco or Redwood City

• 4+ years experience in delivering AI/ML systems and services in a production setting for cloud-native applications
• Experience with distributed compute frameworks
• Experience with the modern AI stack, including improving LLM applications through evals, prompting, and agent scaffolding
• Ability to design and build efficient data storage, compute, and retrieval systems for AI/ML tasks
• Strong communication and coding skills with emphasis on designing for scale and robustness
• Experience owning the delivery of large multi-person projects

• 8+ years of professional software engineering experience
• Experience with architecting and developing production web-scale systems (monitoring, telemetry, performance, reliability, triage and debug)
• Strong development and debugging skills in Python
• Experience with expert data annotation projects
• Experience developing evaluations and environments for complex multi-turn and multi-tool AI systems