Site Reliability Engineer; enhance production observability and incident response using Dynatrace, Splunk

Our financial services client is seeking a Site Reliability Engineer (2+ years) to enhance production observability and incident response using Dynatrace, Splunk, Power BI, and Google Cloud (GCP) Monitoring-

Location:

Toronto/Hybrid- 3 days in-office

Anchor days:
Flexible

Contract Duration: 07/01/ to 06/30/ (Possibility of extension)

Schedule

Hours:

9am-5pm Monday-Friday; standard 37.5 hrs/week

Story Behind the Need

• Business group:
Client Engineering - Mobile and Web

• Project:
Onboarding and Mobile Applications

We’re looking for an SRE with deep experience in production observability and incident response to raise the reliability and transparency of our customer-facing services. You will own the end-to-end observability stack across Dynatrace, Splunk, Power BI, and Google Cloud (GCP) Monitoring, drive proactive detection and reduction of toil, and lead major incident response. This role focuses on operational excellence and service health and NOT platform engineering or Dev Ops provisioning.

Reason
:
This is a 1-year temporary contract position for FTE maternity leave

Candidate Value Proposition

The successful candidate will contribute to a high availability and application stability project.

RESPONSIBILITIES
:

• Design and maintain end-to-end monitoring for critical services using Dynatrace (APM, Real User Monitoring, Synthetic, Davis AI, Smartscape) and GCP Cloud Monitoring (metrics, alerting policies, SLOs/SLIs, uptime checks, dashboards).

• Build service maps, dependency models, and problem detection in Dynatrace; tune Davis AI problem rules and reduce alert noise through thresholds, baselining, and tagging.

• Implement SLOs/SLIs with error budgets; continuously review burn rates and align alerting to customer impact.

• Partner with application teams to instrument code paths (e.g., Dynatrace One Agent), trace distributed transactions, and validate golden signals (latency, traffic, errors, saturation).

Logging, Analytics & Insights (Splunk, Power BI)

• Create and optimize Splunk data models, indexes, sourcetypes, ingestion pipelines, and SPL searches; build actionable dashboards for NOC/SRE/Engineering.

• Develop operational analytics and executive reporting in Power BI (data modeling, DAX/Measures, scheduled refresh) to track reliability KPIs, incident trends, MTTR/MTTD, SLO compliance, and capacity signals.

• Establish governance for data quality, field extractions, and retention to ensure fast, accurate investigations.

Incident Management & Problem Management

• Lead incident response (Sev1/Sev2): run bridges, coordinate SMEs, communicate status/timelines, drive mitigation and customer updates.

• Maintain runbooks, decision trees, and standard operating procedures; ensure blameless post-incident reviews (PIRs) with clear RCA, corrective actions, and preventative measures.

• Track and close problem tickets tied to recurring failure modes; verify effectiveness of fixes via metrics and error budgets.

Reliability Engineering & Automation (Light Coding)

• Use light coding/scripting to automate recurring tasks: alert tuning, data enrichment, log parsing, playbook triggers, service health checks.

• Build small utilities or bots for on-call workflows (e.g., auto-triage, context gathering, incident timelines).

• Contribute to observability standards and best practices (naming, tags, SLIs, alert policies), and mentor teams on instrumenting for reliability.

Candidate Requirements/Must Have

Skills:

• 2+ years of experience as Site Reliability Engineer (SRE)

• 2+ years of experience with Production Operations/Observability with Dynatrace and Splunk in high-availability environments.

• Hands-on recent experience with GCP operations:
Cloud Monitoring, Cloud Logging, Alerting Policies, Uptime Checks, SLOs/SLIs; familiarity with Error Reporting/Trace is a plus.

• Strong SPL (Splunk) and Dynatrace (APM/RUM/Synthetic) expertise—including alert design, dashboards, and noise reduction.

• Proven incident commander experience for Sev1/Sev2 with clear comms, stakeholder management, and PIR facilitation.

• Solid understanding of service reliability concepts: golden signals, SLOs/error budgets, capacity and saturation, graceful degradation.

•…