Lead Data Engineer

Lead Data Engineer

Location:

77380
Duration: 6 Month Contract-to-hire
Work Authorization: US Citizens and Green Card Holders ONLY. This role will have access to federal government information. C2C and third‑party candidates are ineligible.

We are seeking a highly motivated Lead Data Engineer with a passion for data modeling, modern data architecture, and cloud‑native engineering practices. This role is responsible for leading the technical design and implementation of enterprise data platforms while remaining hands‑on in development. The ideal candidate will own solution architecture, technical implementation, delivery planning, estimations, and technical leadership while mentoring engineering team members.

This individual will design scalable, reliable, and high‑performance data platforms using a Databricks‑native architecture built on Delta Lake, Unity Catalog, Delta Live Tables (DLT/Lakeflow Declarative Pipelines), and DBT.

This position partners closely with Product, QA, Project Management, and Business stakeholders to deliver trusted, analytics‑ready data products while proactively managing technical risks, dependencies, and delivery timelines.

• Own the end‑to‑end technical data engineering for enterprise data platforms.
• Lead technical implementation while remaining hands‑on with development.
• Provide delivery estimates, sprint planning input, and technical guidance to engineering teams.
• Mentor and coach data engineers while establishing engineering best practices.

• Design and maintain enterprise dimensional data models that support scalable reporting and analytics.
• Optimize Gold‑layer Delta tables and dimensional models to minimize downstream Power BI DAX complexity through upstream data transformations.
• Apply best practices in data warehousing, semantic modeling, and modern lakehouse architecture.

• Design and develop modular, reusable, metadata‑driven ELT pipelines using Databricks, PySpark, Delta Live Tables (Lakeflow Declarative Pipelines), Unity Catalog, and DBT.
• Implement scalable orchestration patterns using Databricks‑native architecture.
• Build robust, maintainable data pipelines following engineering best practices.

• Build high‑performance transformations using SQL, PySpark, and DBT.
• Implement data quality validations, schema evolution strategies, and automated lineage.
• Develop scalable transformation frameworks that support reusable engineering patterns.

• Design and implement metadata‑driven frameworks, code generation solutions, and agentic engineering patterns to improve engineering productivity and standardization.
• Promote reusable architecture patterns across data engineering initiatives.

• Partner closely with QA teams to define testing strategies, validation criteria, and automated testing for data accuracy, completeness, and reliability.
• Ensure robust quality controls are incorporated throughout the engineering lifecycle.

• Proactively communicate technical progress, sprint status, delivery timelines, project dependencies, and technical risks to Project Management.
• Raise technical blockers early, elevate issues appropriately, and proactively identify delivery risks without requiring micromanagement.

• Integrate data engineering workflows with Git and Azure Dev Ops for source control, automated testing, and continuous deployment.

• Collaborate with analytics, BI, and business stakeholders to ensure data models are optimized for reporting and self‑service analytics.

• Continuously optimize pipeline performance, storage efficiency, and…