Senior Data Engineer

Senior Data Engineer

Salary: $ – $

Division & Section: Technology Services, Office of the Chief Technology Officer

Job Type & Duration: Full-time, 1 Permanent Vacancy

Shift Information: Monday to Friday, 35 hours per week

Affiliation: Non-Union

As a Senior Data Engineer at the City of Toronto, you will have the opportunity to work on cutting‑edge data solutions that directly impact the lives of Toronto's residents. You'll be part of a team driving the city's digital transformation, working on projects that enhance city services and operations through innovative data utilization. You'll work in a collaborative environment that values your expertise and provides opportunities for professional growth.

If you're passionate about leveraging data and AWS technologies to create meaningful change, we encourage you to apply and be part of our mission to build a smarter, more connected Toronto.

Reporting to the Manager, Data Integration & Access, the Senior Data Engineer will join our Enterprise Data Platform team, being a vital partner in supporting the design, development, and implementation of our Enterprise Data Platform.

• AWS Expertise: Utilize a wide range of AWS services to build and maintain scalable, secure, and efficient data infrastructure. Key services include S3, Redshift, Kinesis, EMR, Glue, Data Zone, Lake Formation, and Cloud Formation.
• Data Pipeline Development: Design, implement, and maintain robust ETL/ELT processes using tools such as AWS Glue, DBT (Data Build Tool), and Apache Spark.
• Data Mesh Implementation: Contribute to the implementation of a data mesh architecture, enabling decentralized, domain‑oriented data ownership and management.
• Infrastructure as Code: Develop and maintain infrastructure as code using Terraform or AWS Cloud Formation to automate and streamline the deployment of cloud resources.
• Data Processing: Utilize Python and Apache Spark for large‑scale data processing, transformation, and analysis.
• Data Modeling: Design and implement efficient data models to support analytics, machine learning, and reporting needs.
• Streaming Solutions: Develop and maintain both batch and real‑time data streaming solutions using technologies such as AWS Kinesis.
• Data Governance: Implement and adhere to data governance policies to ensure data quality, privacy, and compliance with regulations.
• Platform Enhancement: Work with technologies such as Databricks and Snowflake to enhance the capabilities of the data platform.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide tailored solutions.
• Documentation and Knowledge Sharing: Create and maintain comprehensive documentation for data processes, pipelines, and models. Share knowledge with team members and contribute to the team's overall growth.