More jobs:
Senior AI Observability Engineer
Job in
Toronto, Ontario, C6A, Canada
Listed on 2026-10-25
Listing for:
Aimlroles
Full Time
position Listed on 2026-10-25
Job specializations:
-
Software Development
AI Engineer (Applied/Software), AI Reliability/ Performance Engineer, AI QA / Validation Engineer
Job Description & How to Apply Below
Build AI observability systems at BMO in a hybrid role, using Python, telemetry, and evaluation methods for production models and agents.
As a Senior AI Engineer, you will design systems that trace LLM calls, monitor token usage, detect drift, and evaluate autonomous workflows. The work includes machine learning, deep learning, statistical analysis, and specialized technical support. You will help govern agent behavior and diagnose non-routine production problems.
Key Responsibilities:- Design and maintain AI model observability systems
- Trace agent trajectories, prompts, tools, and outputs
- Build automated evaluations for relevance and hallucinations
- Monitor drift, latency, errors, and data quality
- Define guardrails and self-remediation boundaries
- Bachelor's degree in Computer Science or Engineering
- Seven to ten years of Site Reliability Engineering experience
- Experience in Artificial Intelligence Observability Engineering
- Python, SQL, LogQL, or PromQL proficiency
- Experience with Prometheus, Grafana, or Datadog
Apply AI engineering, observability, and evaluation expertise to support reliable agent workflows at BMO.
Position Requirements
10+ Years
work experience
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×