ML Engineer

About Rust Labs

We’re building the data layer for frontier AI. Rust Labs is a high-throughput annotation and evaluation platform used by AI labs to produce training data, RLHF preference signals, and expert evaluations across text, image, code, and multimodal domains. We’re early, well-funded, and working directly with research teams at top labs.

You’ll own the technical core of the platform — the pipelines that turn raw tasker output into clean, model-ready training data. This is a hands‑on IC role with significant ownership: you’ll design annotation schemas with customers, build evaluation infrastructure, write the tooling that ensures data quality at scale, and sit at the intersection of ML research and operations.

• Design and ship annotation pipelines (RLHF, SFT, eval, red‑teaming) end-to‑end — schema design, tasker UX, quality controls, aggregation, delivery to customers.
• Build evaluation infrastructure: automated checks, LLM-as-judge systems, calibration tooling, and inter‑annotator agreement metrics.
• Work directly with research teams at AI labs to understand their data needs and translate them into shippable annotation products.
• Build the internal tooling that lets a distributed tasker workforce produce gold‑standard data at scale — onboarding flows, qualification tests, payout logic, quality dashboards.
• Contribute to research artifacts (datasets, evals, papers) when relevant work ships.

• 2+ years of professional experience as an ML engineer, research engineer, or similar role. Strong Python, comfort with modern ML frameworks (PyTorch, Hugging Face).
• You’ve shipped ML systems to production — you’ve debugged training pipelines, you know what “data quality” actually means in practice, you understand the failure modes of LLM evaluation.
• Familiarity with RLHF, SFT, or evaluation methodologies is a strong plus.
• You can write clean code, move fast, and talk directly to customers.
• Bonus: prior experience at an AI lab, data platform, or annotation company. Bonus: open-source contributions.