Data Engineer/Data Architect; Azure/Databricks),leading architecture,design Job Toronto area,Ontario Canada,IT/Tech

Position: Staff Data Engineer/Data Architect (Azure / Databricks), you will be responsible for leading the architecture, design

12 month contract

2 days on site in Oshawa

Job Overview

As a Staff Data Engineer/Data Architect, you will be responsible for leading the architecture, design and delivery of scalable data pipelines and data products which enable innovative, customer-centric digital experiences. You will be working as part of a cross-discipline agile team who helps each other solve problems across all business areas. You will be a thought leader and subject matter expert on the data lakehouse, data warehousing and modeling activities for the team and use your influence to ensure that the team produces best-in-class data solutions that leverage repeatable, maintainable, and well-documented design patterns.

You will employ best practice in development, security, accessibility and design to achieve the highest quality of service for our customers.

Lead the architecture, design and oversee implementation of modular and scalable data ELT/ETL pipelines and data infrastructure leveraging the wide range of data sources across the organization

Design curated common data models that offer an integrated, business-centric single source of truth for business intelligence, reporting, and downstream system use

Work closely with infrastructure and cyber teams to ensure data is secure in transit and at rest

Create, guide and enforce code templates for delivery of data pipelines and transformations for structured, semi-structured and unstructured data sets

Develop modeling guidelines that ensure model extensibility and reuse by employing industry standard disciplines for building facts, dimensions, bridge, aggregates, slowly changing dimensions and other dimensional and fact optimizations

Establish standards database system fields, including primary and natural key combinations that optimize join performance in a multi-domain, multiple subject area physical (structured zone) and semantic model (curated zone)

Ensure model extensibility by employing industry standard disciplines for building facts, dimensions, bridge, aggregates, slowly changing dimensions and other dimensional and fact optimizations

Transform data and map to more valuable and understandable semantic layer sets for consumption, transitioning from system-centric language to business-centric language

Collaborate with business analysts, data scientists, data engineers, data analysts and solution architects to develop data pipelines to feed our data marketplace

Introduce new technologies to the environment through research and POCs, and prepare POC code designs that can be implemented and productionized by developers

Work with tools in the Microsoft Stack;
Azure Data Factory, Azure Data Lake, Azure SQL Databases, Azure Data Warehouse, Azure Synapse Analytics Services, Azure Databricks, Microsoft Purview, and Power BI

Work within the agile SCRUM work management framework in delivery of products and services, including contributing to feature & user story backlog item development, and utilizing related Kanban/SCRUM toolsets

Document as-built architecture and designs within the product description

Design data solutions that enable batch, near-real-time, event-driven, and/or streaming approaches depending on business requirements

Design & advise on orchestration of data pipeline execution to ensure data products meet customer latency expectations, dependencies are managed, and datasets are as up-to-date as possible, with minimal disruption to end-customer use

Ensure that designs are implemented with proper attention to data security, access management, and data cataloging requirements

Approve pull requests related to production deployments

Demonstrate solutions to business customers to ensure customer acceptance and solicit feedback to drive iterative improvements

Assist in troubleshooting issues for datasets produced by the team (Tier 3 support), on an as-required basis

Guide data modelers, business analysts and data scientists in the build of models optimized for KPI delivery, actionable feedback/writeback to operational systems and enhancing the predictability of machine learning models and experiments

Qualifications

Requires an extensive knowledge in designing a data model to solve a…

Data Engineer​/Data Architect; Azure​/Databricks), leading architecture, design

Data Engineer/Data Architect; Azure/Databricks), leading architecture, design