Data Engineer – Microsoft Fabric & Cloud Data Platforms; Remote from India)

Thinkgrid Labs designs and builds custom web, mobile, cloud, data, and AI solutions. Our team brings together software engineers, architects, data engineers, and UI/UX designers to deliver reliable systems for clients around the world.
We’re expanding our data practice and building a modern data platform for a US health insurer
, with Microsoft Fabric as the primary platform
. You’ll design, build, and operate reliable data pipelines—from source ingestion and transformation to analytics-ready lakehouse and warehouse models. Hands-on Microsoft Fabric experience is strongly preferred, and we welcome engineers with strong Databricks or comparable cloud data platform experience who are ready to apply their skills in Fabric.
Job Title: Data Engineer – Microsoft Fabric & Cloud Data Platforms
Location: Remote
Working Hours: 2 PM IST to 11 PM IST
Experience

Required:

6–10 years in data engineering, including 2+ years working with cloud data platforms
Education: Bachelor’s or Master’s degree in Computer Science, Health Informatics, or Business
Preferred

Certifications:

Relevant Microsoft Fabric/Azure, Databricks, or equivalent data engineering certifications

Who are you?

• Strong in SQL and Data Modeling: You work confidently with complex relational databases and design practical data models for integration, reporting, and analytics.
• Experienced with Cloud Data Platforms: You’ve built production solutions on Microsoft Fabric, Databricks, or comparable platforms. Experience with Fabric Mirroring, One Lake, Lakehouse/Warehouse, and Fabric Data Factory is particularly valuable.
• A Reliable Pipeline Builder: You design ingestion and transformation pipelines that handle incremental loads, CDC, schema changes, and recovery from failures.
• Comfortable with Python and Spark: You use code and notebooks to process data, implement reusable workflows, and validate results.
• Focused on Data Quality: You build checks, reconciliation, and clear data contracts into your work so downstream teams can trust the data.
• Operationally Minded: You monitor pipeline health, troubleshoot production issues, document runbooks, and balance performance with cost.
• Security Conscious: You handle sensitive data responsibly, including PII/PHI, and apply appropriate access controls and governance practices.
• What will you be doing?
• Build Data Pipelines: Develop and maintain ingestion and ETL/ELT pipelines across relational databases, files, APIs, and other sources, primarily using Microsoft Fabric.
• Develop Lakehouse and Warehouse Solutions: Organize data across raw, refined, and curated layers, using One Lake and Fabric Lakehouse/Warehouse to support reliable downstream consumption.
• Transform and Model Data: Build reusable transformations and analytics-ready models that translate source data into useful, well-documented datasets.
• Manage Incremental Data and Change: Implement CDC, replication, or watermark-based loading as appropriate. Handle deletes, late-arriving data, backfills, and schema evolution safely, using Fabric Mirroring where suitable.
• Orchestrate Resilient Workflows: Manage dependencies, scheduling, retries, idempotency, and failure recovery using Fabric Data Factory, notebooks, and appropriate orchestration tools.
• Ensure Quality and Observability: Implement data validation, reconciliation, logging, metrics, lineage, and alerts to maintain data accuracy and pipeline reliability.
• Optimize Performance and Cost: Tune queries, Spark workloads, partitioning, file sizes, and compute usage to meet delivery expectations efficiently.
• Collaborate and Document: Work with platform architects, security teams, analysts, and business stakeholders on requirements, governance, and delivery. Maintain pipeline documentation, data definitions, SLAs, and operational…