Lead Data Engineer

Lead Data Engineer (Frisco, TX & Bellevue, WA)

We are seeking a Lead Data Engineer to design and build a scalable, high‑quality data platform that ingests data from multiple sources, ensures data quality and governance, and delivers near real‑time insights (15‑minute SLA) through Power BI / Microsoft Fabric dashboards.

This role will provide technical leadership and drive end‑to‑end data engineering architecture and delivery.

• Design and implement end‑to‑end data platform architecture (ingestion, processing, storage, serving).
• Define batch and near real‑time data pipelines ensuring low‑latency and high reliability.
• Make technology decisions across Databricks, Delta Lake, Snowflake, and Azure ecosystem.

• Build scalable pipelines using Databricks / Spark for large‑scale processing.
• Use Delta Lake for ACID‑compliant data storage.
• Use Snowflake for data warehousing and analytics.
• Implement ETL/ELT pipelines with strong data modeling practices.

• Design and implement real‑time pipelines using Kafka / Azure Event Hubs.
• Ensure data freshness within ~15‑minute SLA.
• Enable incremental processing and efficient data updates.

• Establish data quality frameworks (validation, completeness, consistency checks).
• Implement monitoring, alerting, and data observability.
• Define and enforce data governance, lineage, and metadata standards.

• Enable optimized data layers for Power BI / Microsoft Fabric dashboards.
• Design semantic models and curated data layers for business consumption.
• Ensure consistent, accurate, and high‑performance reporting.

• Optimize pipelines and storage for large‑scale datasets (TB/PB).
• Ensure low‑latency query performance and efficient compute usage.
• Implement partitioning, indexing, caching, and optimization strategies.

• Lead and mentor a team of data engineers.
• Collaborate with Technical Product Managers, BI teams, and business stakeholders.
• Drive best practices in coding, architecture, and delivery.
• Manage technical risks, dependencies, and roadmap execution.

• Strong experience in data engineering and platform architecture (Lead level).
• Expertise in Databricks, Spark, Delta Lake, Snowflake or similar cloud data warehouses.
• Hands‑on with streaming technologies (Kafka / Event Hubs).
• Strong knowledge of data modeling, ETL/ELT, and pipeline design.
• Experience with data quality frameworks and monitoring tools.
• Familiarity with Power BI / Microsoft Fabric.
• Strong programming skills (Python, SQL).
• Experience with Azure ecosystem (ADF, ADLS, AKS – preferred).
• Nice to have:
  Experience with real‑time analytics platforms.
• Exposure to data governance / MDM frameworks.
• Familiarity with CI/CD and Dev Ops practices for data platforms.

• Own and deliver a robust, scalable data platform.
• Ensure high data quality and near real‑time availability (15‑minute SLA).
• Drive standardization, reusability, and performance optimization.
• Enable business‑ready, trusted data for analytics and decision‑making.

Build and scale a modern data platform that delivers trusted, near real‑time insights, enabling faster decisions and powering analytics across the organization.