ML Lead, AI Data Labeling

About NewtonX

Newton

X is a B2B insights company trusted by the world's most innovative companies to make high-stakes decisions with confidence. We combine a verified network of business professionals with AI-powered research tools to deliver research intelligence faster, more precise, and more defensible than traditional methods. Our clients include Google, Microsoft, Tik Tok, Door Dash, Stripe, and Coinbase. Our research has been cited by Fortune, Forbes, Tech Crunch, Adweek, and the Wall Street Journal.

Newton

X has raised $47M from investors including Two Sigma Ventures, Third Prime, XFund, and Citi Ventures.

About the Role

AI buyers have changed. From mid-market SaaS companies fine-tuning open-source models to Fortune 500 enterprises building internal AI platforms to frontier AI labs running large-scale evaluations, the question is no longer “is AI useful” but “how do we evaluate whether our AI works?” Every one of these buyers needs structured, expert-grounded evaluation data and domain-specific benchmarks. Almost none of them can build it themselves.

That is the opportunity as ML Lead. Rolling up directly to the VP of Commercial, you are the technical counterpart to ML and product teams across our client base, spanning growth-stage AI companies, enterprise AI platforms, and frontier research labs. You sit in their working sessions, hold your ground on technical specifics (eval design, statistical significance, contamination concerns, inter-annotator reliability), translate what they actually need into concrete operational specs, and partner with our recruiting and ops lead to build the expert pipelines that produce defensible data.

You also build. Beyond bespoke client work, you own the design and development of Newton

X domain benchmarks across high-value verticals (finance, legal, healthcare, and others as we expand). These become both syndicated products and methodological proof points that move us up the client sophistication curve.

And you sell, lightly but meaningfully. You are on client calls. You hear gaps. You spot opportunities other vendors miss. You bring those back, shape them into pitches, and partner with Commercial to expand accounts.

Client Technical Partnership

• Serve as the primary technical point of contact for ML, applied science, and product teams at our AI-focused clients across the maturity spectrum, from emerging AI companies to enterprise platforms to frontier labs.
• Hold your own in technical conversations: eval design, dataset construction, contamination risk, statistical power, inter-annotator agreement, RLHF data quality, agentic evaluation, red-teaming methodology.
• Translate ambiguous technical requirements into concrete operational specs: target expert profiles, screener trees, task design, annotation rubrics, quality control protocols, statistical sampling plans.
• Calibrate depth to the audience. A Series B AI startup and a frontier lab need different conversations. You can run both.

Domain Benchmark Development

• Design and build domain benchmarks for Newton
  
  X-owned domains in high-value verticals. Initial targets: finance (markets, accounting, regulatory), legal (contracts, case reasoning, jurisdictional), healthcare (clinical reasoning, diagnostic, regulatory). Additional verticals as the business expands.
• Architect benchmark structure: task taxonomy, difficulty distribution, expert involvement model, evaluation rubrics, scoring protocols, baseline scoring against frontier models.
• Recruit and calibrate the domain experts who write, validate, and grade benchmark tasks. Work with our recruiting and ops lead to operationalize at scale.
• Publish methodology papers, technical reports, and leaderboards that make Newton
  
  X benchmarks the reference standard in their verticals.

Operationalization with Newton

X Recruiting and Ops

• Work directly with our full-time recruiting and operations lead to convert client and benchmark requirements into operational specs: expert profiles, screeners, task interfaces, annotation workflows, QC sampling rates, and fielding timelines.
• Calibrate the recruiting team on what “good” looks like for each engagement. Run alignment…