Azure Data Engineer

We are seeking an Azure Data Engineer with strong, hands-on Microsoft Fabric experience to build and operate a scalable middle-layer data and integration platform. The role focuses on ingesting data from diverse sources, implementing transformation and business rules, and delivering curated, consumption-ready datasets/models for analytics and downstream applications.

Work on the fabrics platform to design and implement robust data solutions, including One Lake architecture for efficient data storage and processing.

Design, develop, and maintain scalable ingestion and transformation pipelines using Microsoft Fabric (Data Factory in Fabric / Pipelines), ADF/Synapse Pipelines, One Lake storage patterns, PySpark, Python, and SQL across structured and unstructured data.

Develop pipelines that ingest data from external APIs on a scheduled basis and initiate end-to-end downstream processing, supporting one or multiple daily runs through to curated and consumption-ready layers.

Integrate data from cloud and on-prem sources including databases, third-party systems, files, and REST/SOAP APIs (auth, throttling, pagination, retries, and error handling).

Build curated layers and consumption-ready models; implement incremental and batch processing logic; apply data modelling and transformation best practices aligned to reporting/analytics needs.

Develop and optimize complex queries, stored procedures, views, and datasets for efficient analytics and reporting; partner with analytics teams to meet performance SLAs.

Tune Spark jobs, ADF data flows and SQL workloads (partitioning, caching, parallelism, cluster sizing/configs) to improve reliability and reduce runtime/cost.

Translate requirements into scalable rules (validation, eligibility, availability calculations), manage exceptions, audit logging, and ensure data consistency across systems.

Implement automated data quality checks, validation frameworks, reconciliation, and monitoring to ensure trusted datasets.

Implement secure access via Azure AD, Managed Identities, RBAC, least privilege, and secure connectivity to data lake, Fabric/Synapse, and APIs.

Build deployment automation using Azure Dev Ops/Git, promoting code across environments with consistent release practices; support testing and release activities.

Monitor pipelines and jobs using Spark UI and Azure Log Analytics; triage failures, perform root-cause analysis, and improve resiliency/runbooks.

Work closely with architects, platform/Dev Ops engineers, analysts, and data scientists; participate in design sessions and code reviews; operate within Agile/Scrum delivery.

• Microsoft Fabric Work spaces
• One Lake
• Fabric Pipelines / Data Factory in Fabric

• ADLS Gen2
• Blob Storage
• Synapse Analytics
• App Service (as needed)

• Py Spark
• Python
• SQL (T-SQL)

• Git

Terraform (preferred)

• Spark UI

• Experience:
  
  5+ years
• Role Level: Individual Contributor (Senior Engineer)

Symbiotic Digital provides the IT Experts you need that solve problems and get things done. See what the top 2% in their field can do for you. (Symbiotic Digital is a division of Symbiotic Group Inc.)

We serve customers in two ways, we provide:

1) Digital Experts: IT Consulting expertise. Bright Minds That Produce Proven Results

2) Recruitment:
Find The Right People You Need

Serving IT & Business leaders, Symbiotic Digital provides a business system that enable us to provide the Expert Technical staff needed to solve challenging business and IT problems. IT leaders in the most reputable organizations in Canada have relied on Symbiotic Digital to help them staff difficult assignments with proven Experts uniquely evaluated by our proven QMS
- Quality Management System Staff Development Model. We are a 100% Indigenous owned company.