Principal Data Scientist

Location: Remote (US)

Employment type: Full-time

Salary: $173,000–$290,900 per year, depending on location and experience

Additional compensation: Equity awards

A Principal Data Scientist is sought to lead measurement strategy, experimentation and AI evaluation across large-scale technology platforms. The role focuses on measuring the impact of platform investments, evaluating LLM-based and agentic systems, and developing reliable frameworks to connect product performance with customer and business outcomes.

Working closely with Engineering and Product leadership, you will define critical metrics, build causal inference models and establish measurement standards that inform strategic and executive-level decisions.

• Define and implement measurement strategies across technology platforms, from data instrumentation and pipelines to executive reporting.
• Establish key performance indicators, leading and lagging indicators, dashboards and measurement frameworks.
• Design and evaluate LLM-based and agentic systems using structured evaluation methods, including LLM-as-a-judge and automated evaluation frameworks.
• Lead experimentation and causal inference programmes, including A/B testing and alternative methodologies when randomised experiments are not possible.
• Apply methods such as difference-in-differences, synthetic controls, interrupted time series and switchback experiments to measure business impact.
• Connect platform adoption, AI performance and infrastructure investments to customer experience, productivity and revenue outcomes.
• Partner with Engineering teams to define event schemas, instrumentation requirements, data pipelines and measurement infrastructure.
• Identify meaningful performance indicators and challenge metrics that do not demonstrate genuine business or customer value.
• Translate complex analytical findings into clear recommendations for senior executives and business leaders.
• Establish best practices for platform measurement, experimentation and data science across the organisation.

• 10+ years of experience in Data Science, UX Research or Applied Science.
• Experience measuring complex products, platforms or internal technology systems.
• Strong expertise in causal inference, experimentation and A/B testing methodologies.
• Experience designing product metrics, KPIs, OKRs and measurement frameworks.
• Practical experience evaluating complex user-facing systems, including AI-powered or LLM-based products.
• Knowledge of LLM evaluation methods, including LLM-as-a-judge and offline/online evaluation frameworks.
• Strong Python and SQL programming skills, including data modelling and instrumentation.
• Experience working closely with Engineering teams on event schemas, data pipelines and measurement systems.
• Ability to communicate complex analytical findings and influence senior executives and technical stakeholders.
• Strong independent problem-solving skills and the ability to lead complex analytical initiatives.

• PhD with 4+ years, Master's degree with 6+ years, or Bachelor's degree with 8+ years of relevant experience in Statistics, Econometrics, Data Science, UX Research, Machine Learning, Operations Research or a related quantitative field.
• Experience measuring production conversational AI, LLM-based or agentic systems.
• Familiarity with evaluation tools such as Braintrust.
• Experience measuring AI adoption, productivity improvements, quality and return on investment.
• Experience with platform-as-a-product measurement models.
• Advanced knowledge of quasi-experimental and causal inference techniques.
• Base salary of $173,000–$290,900 per year, depending on location.
• Equity awards.
• Remote working within the United States, subject to location eligibility.