Skip to content
×

Agentic AI Evaluation Engineer

Job in Toronto, Ontario, C6A, Canada
Listing for: Equifax Canada
Full Time position Listed on 2026-11-20
Job specializations:
Salary/Wage Range or Industry Benchmark: 110000 - 150000 CAD Yearly CAD 110000.00 150000.00 YEAR

Job Description & How to Apply Below

Build reliable autonomous workflows at Equifax in Toronto using Python, LLM evaluation, and observability for production AI systems.
This newly created full-time role focuses on Agentic AI Evaluation and Tuning Engineering. You will test multi-agent architectures, audit decision-making logic, and optimize tool use. The work connects automated testing, prompt tuning, orchestration frameworks, and production guardrails.

Key Responsibilities:

Curate Golden Sets of documents, queries, and agent trajectories
Automate continuous evaluation pipelines for accuracy, latency, and token spend
Audit multi-step reasoning and identify deviations from intended logic
Optimize prompts, context windows, and few-shot examples
Monitor deployed agents for drift, injections, loops, and hallucinations
Requirements:
Three-plus years in quality engineering, test automation, or data engineering
Recent focus on LLM testing, prompt tuning, or orchestration
Hands-on Lang Graph, ADKs, or specialized internal SDK experience
Python-heavy automated testing and tracing experience
Understanding of JSON schemas, function calling, and structured outputs
Apply Agentic AI evaluation, Python automation, and observability expertise to strengthen Equifax autonomous systems in Toronto.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction

Search for further Jobs Here:

SEARCH JOBS: 10+ Million Listings
(Try combinations for better Results! Or enter less keywords for broader Results)
Education Level
Experience Level (years)
Posted in last:
Salary