Data Engineer

Job Overview

We are seeking a Data Engineer to help design, build, and scale an enterprise data platform on Microsoft Fabric and complementary Azure data services. The role focuses on delivering high‑quality, insight‑ready data products using a Medallion architecture that supports analytics, reporting, and AI‑driven decision‑making. You will process both structured and unstructured data—documents, PDFs, scanned files, and text‑heavy content—to transform them into trusted, auditable datasets for analytics and AI solutions.

Hybrid work guideline requires teams to come to the office a minimum of 4 days per week.

• Design, build, and maintain end‑to‑end data pipelines using Microsoft Fabric (Data Pipelines, Lakehouse/Warehouse, Notebooks) and, where appropriate,
  Azure Data Factory
  .
• Implement scalable data transformations aligned to a Medallion architecture, producing curated, analytics‑ready datasets.
• Support environment separation (Dev / UAT / Prod), Git‑enabled development, and CI/CD‑style deployment practices.

• Build ingestion and processing workflows for unstructured and semi‑structured content, including PDFs, scanned documents, reports, agreements, attachments, and text‑heavy or non‑tabular source files.
• Enable document‑understanding pipelines that support text extraction (including OCR where required), document classification, attribute extraction, and standardization/enrichment of extracted data.
• Integrate unstructured data outputs with structured datasets to create a unified, trusted analytical view.
• Design solutions with strong auditability, traceability, and exception handling to ensure confidence in downstream insights.

• Translate business questions into well‑designed data structures that support analysis, comparison, and trend detection.
• Create curated datasets and models optimized for Power BI
  , semantic models, and AI consumption.
• Ensure clarity in data grain, metric definitions, and relationships so insights are explainable and defensible.

• Implement data quality checks, reconciliation logic, and validation rules across pipelines.
• Monitor pipeline health, data anomalies, and processing exceptions.
• Contribute to common engineering standards, reusable patterns, and clear documentation to support scale and consistency.

• Work closely with analytics, AI, and business teams to co‑design data products that meet real business needs.
• Support federated teams by creating reusable components, templates, and best‑practice guidance.
• Communicate technical choices and data limitations clearly to both technical and non‑technical stakeholders.

• 4–7+ years of experience in data engineering or analytics engineering roles.
• Hands‑on experience with Microsoft Fabric and/or a strong background in Azure Data Factory /
  Azure data platforms, with the ability to apply those skills in a Fabric‑first environment.

• Strong SQL for transformation, modeling, and performance optimization.
• Strong Python experience (PySpark preferred).
• Proven experience implementing Medallion‑style data architectures for analytics.
• Experience delivering production‑grade, reliable, and maintainable data pipelines.

• Demonstrated experience processing unstructured data (documents, PDFs, scanned files, text content).
• Familiarity with OCR
  , text extraction, or document parsing concepts.
• Experience integrating extracted document data into analytical datasets and reconciling it with structured sources.

• Deep hands‑on experience with Microsoft Fabric Lakehouse, Warehouse, Notebooks, and Pipelines.
• Experience with Git‑based development and CI/CD practices for data platforms.
• Exposure to data designs that support AI‑enabled analytics or advanced insight generation.
• Experience working within enterprise governance, security, and compliance constraints.

• Reliable, well‑monitored data pipelines supporting both structured and unstructured sources.
• High‑trust,…