Senior Data Engineer – Azure & Microsoft Fabric

Title:

Sr. Data Engineer

Location:

Deerfield Beach, FL (Hybrid 3/2)

Duration: 6 Months+

Compensation: $75/hr -$80/hr

Work Requirements: U.S. Citizen, GC Holders, or Authorized to Work in the U.S.

The Senior Data Engineer designs, builds, tests, and supports the data pipelines and data products

behind our analytics, reporting, and machine learning work on Azure and Microsoft Fabric. The role

focuses on data engineering: ingesting, transforming, and modeling structured and unstructured data

from sources such as Azure Synapse, Azure SQL, SQL Server, and ADLS Gen
2. That data lands in

Fabric Lakehouse and Warehouse on One Lake through Azure Data Factory, Fabric Data Factory

pipelines, Dataflow Gen2, and PySpark notebooks. The engineer will also design and run machine

learning (ML) pipelines with the Data Science team, automate builds and releases through Azure Dev Ops

pipelines, and set up supporting Azure resources with Bicep templates. They will apply secure data

practices (Managed Identity, Microsoft Entra , Key Vault, One Lake security, and Microsoft Purview) and

build data quality, lineage, and governance into every pipeline. This role works closely with Data

Scientists, Solution Architects, Software Engineers, Dev Ops Engineers, QA Engineers, Project Managers,

Information Security Engineers, and Delivery Managers.

• 10% Data Exploration and Design (Source Analysis, Data Modeling, Security, Quality, Performance)
• 40% ETL/ELT Pipeline Development (Azure Data Factory, Fabric Data Factory, Dataflow Gen2, Fabric Notebooks, Unit Testing)
• 20% ML Pipeline Design and Development (Data Preparation, Feature Engineering, Training and Batch Scoring Pipelines, MLflow)
• 20% Dev Ops and Infrastructure as Code (Azure Dev Ops CI/CD Pipelines, Bicep Templates, Fabric Workspace Promotion, Release Management)
• 10% Data Operations and Governance (Monitoring, Purview Lineage, Access Control, Cost and Performance Optimization)

• Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory and Microsoft Fabric Data Factory (pipelines, Copy activity, Dataflow Gen2). These pipelines pull data from Azure Synapse, Azure SQL, SQL Server, APIs, and file-based sources.
• Build a medallion architecture (bronze, silver, gold) in Fabric Lakehouse and Warehouse on One Lake using Delta Parquet tables, with shortcuts and Mirroring where they fit.
• Write data transformations in PySpark, Spark SQL, and T-SQL in Fabric Notebooks, and build reusable, parameterized, metadata-driven pipeline frameworks.
• Replace legacy SAS and file-based ETL processes (CSV exports, file shares) with governed ADF and Fabric pipelines that include alerting, retries, and the ability to re-run.
• Design and develop ML pipelines with the Data Science team: data preparation, feature engineering, model training, and batch scoring (Fabric PREDICT). Publish model outputs to Lakehouse tables for Power BI (Direct Lake) reporting.
• Manage the model lifecycle with MLflow experiment tracking and the model registry, and help move Azure Machine Learning workloads to Fabric Data Science.
• Build and maintain Azure Dev Ops CI/CD pipelines (YAML) for ADF, Fabric items, notebooks, SQL objects, and ML assets. This includes Git integration, branching strategy, pull-request validation, and promotion between Dev and Prod work spaces. Write and maintain Bicep templates that set up supporting Azure resources the same way in every environment (Data Factory, ADLS Gen2, Key Vault, managed identities, private endpoints, and networking).
• Build dimensional (star schema) data models and Fabric semantic models for analytics and reporting.
• Apply data security and compliance controls:
  Managed Identity and Entra , Key Vault secrets, One Lake and workspace roles, row- and column-level security, data masking, and sensitivity labels.
• Build data quality checks, lineage, and cataloging (Microsoft Purview) into every data-movement process.
• Monitor, troubleshoot, and tune pipelines, Spark jobs, and queries for performance and capacity cost using the Fabric Monitoring Hub and Azure Monitor.
• Work with stakeholders to turn business requirements into technical specifications, and keep design specifications and component documents up to date.
• Keep up with the Azure and Microsoft Fabric roadmap and recommend generally available features and best practices to adopt.

• Must be a team player with a…