Data Scientist

About This Opportunity

Are you passionate about ML model quality and building trust in AI systems? If so, this might be the role for you! We’re looking for a Data Scientist to join our growing ML & AI practice. You will play a key role in ensuring the reliability, accuracy, and performance of our machine learning models, from evaluation framework design to ongoing production monitoring.

Our Strategy and Analytics Department empowers the company by turning data into actionable insights and supporting various teams. It serves as a central hub that drives business outcomes across our ecosystem.

Reporting to the Director, Data Science, the Data Scientist will be a core contributor to the quality and reliability of our ML and AI systems. Your primary focus will be ML model validation and monitoring, designing and executing MCP (Model Context Protocol) evaluations, and building regression test suites that give the team confidence at every stage of the model lifecycle.

• Design, implement, and maintain ML model validation frameworks, including custom evaluation metrics, loss functions, and statistical tests, to ensure model quality before and after deployment.
• Build and own regression test suites for ML and LLM models, catching performance regressions and unexpected behaviour across model updates and data drift scenarios.
• Develop and execute MCP evaluations, systematically assessing model capabilities, edge cases, and failure modes across relevant business contexts.
• Monitor models in production using statistical process control, drift detection, and alerting pipelines; proactively surface issues before they impact customers.
• Collaborate with senior data scientists to contribute to the design and refinement of ML model architectures, offering feedback grounded in validation results.
• Document evaluation methodologies, test results, and monitoring runbooks clearly enough that stakeholders across technical and business teams can understand model health.
• Stay current with advancements in LLM evaluation techniques, AI safety, and model observability, and apply emerging best practices to our workflows.
• Communicate findings clearly and concisely to stakeholders, translating model performance signals into actionable recommendations.

• Industry experience in data science, machine learning, or a closely related quantitative field.
• Proficiency in Python and the core DS stack:
  Pandas, Scikit‑Learn, XGBoost, and at least one deep learning framework (PyTorch or Tensor Flow).
• Solid grasp of statistical concepts underpinning model evaluation: bias–variance trade‑off, calibration, confidence intervals, A/B testing, and data drift.
• Experience with LLM evaluation frameworks (e.g. RAGAS, Eleuther AI Eval Harness, or custom LLM eval pipelines).
• Hands‑on experience designing custom evaluation metrics beyond off‑the‑shelf metrics.
• Strong understanding of ML and LLM model architectures.
• High proficiency in SQL for data exploration, feature validation, and debugging model inputs.
• Exceptional attention to detail, treating model validation with the same rigor as software QA.
• Strong written and verbal communication skills; comfortable presenting findings to both technical and non‑technical stakeholders.

• Experience building model evaluation or monitoring solutions within a Snowflake environment.
• Familiarity with Snowpark (Python) for running data transformations and ML workflows directly within Snowflake.
• Exposure to model regression testing patterns in a CI/CD context.
• Familiarity with prompt engineering and evaluation strategies for LLM‑powered features.
• Experience working in a SaaS environment and an appreciation for how model quality translates to customer impact.

Open to candidates based in Edmonton, Toronto, Vancouver, or Kitchener‑Waterloo.

Minimum annual salary of $125,800 CAD, midpoint of $147,900 CAD, and maximum of $170,100 CAD.

• Extended health benefits package with fully paid premiums for both body and mind.
• Matching in RRSP, TFSA or FHSA.
• Stock options.
• Dedicated Talent Development team with coaching, learning, and leadership programs.

We believe diverse teams perform better and that fostering an inclusive workplace is a key part of growing a successful team. We welcome people of diverse backgrounds, experiences, and perspectives. We are an equal opportunity employer and are committed to working with applicants requesting accommodation at any stage of the hiring process.