Sr. Data Platform Engineering

Job Category : Information Technology

Supervisor : Ahmed El Mehalawy

Requisition Number : CACTU
002631

• Posted :
  August 7, 2026
• Full-Time
• On-site

Showing 1 location

Houston
920 Memorial City Way
Ste 300
Houston, TX 77024, USA

Houston
920 Memorial City Way
Ste 300
Houston, TX 77024, USA

Onsite 4 days in office 1 working from home

No fully remote options

Job Summary:

The Data Platform Lead is responsible for implementing, configuring, and governing the technical foundation of the company’s enterprise data Lakehouse on Azure Databricks. This role combines hands‑on data platform ownership with data engineering leadership, ensuring that data from ERP systems, product SaaS PostgreSQL databases, APIs, and other enterprise sources is ingested, modeled, governed, secured, and prepared for analytics, reporting, automation, and AI use cases.

This position serves as the internal technical owner for Databricks platform standards, Bronze/Silver/Gold data architecture, Unity Catalog governance, pipeline design patterns, source‑to‑target mapping standards, data quality implementation, vendor technical review, and production readiness. The role is hands‑on, with responsibility for platform configuration, environment setup, access controls, catalog/schema structure, compute standards, and operational readiness, while also providing technical directions to data engineers, contractors, vendors, and future internal team members through strong standards, practical architecture, and disciplined delivery.

Essential Functions,

Roles and Responsibilities:

Essential duties and responsibilities include the following:

Data Platform & Lakehouse Architecture

• Own the technical architecture for the Azure Databricks Lakehouse, including workspace structure, catalogs, schemas, compute patterns, storage strategy, and environment separation.
• Define and maintain bronze, silver, and gold layer standards, including naming conventions, table ownership, audit columns, refresh patterns, and production readiness criteria.
• Implement and govern Unity Catalog standards for access control, lineage, data classification, catalog/schema organization, and least‑privilege access.
• Partner with cybersecurity, infrastructure, and Dev Ops teams to align Databricks with enterprise identity, networking, secrets management, monitoring, and compliance expectations.
• Establish cost controls, cluster policies, job standards, and usage monitoring to ensure the platform is reliable, scalable, and cost‑effective.
• Design, build, and oversee production‑grade data pipelines using Databricks, Spark, Python/PySpark, SQL, Delta Lake, and approved orchestration patterns.
• Lead ingestion from ERP systems, product PostgreSQL databases, SaaS platforms, APIs, files, and other enterprise data sources into the Lakehouse.
• Define engineering patterns for full loads, incremental loads, CDC where applicable, reprocessing, error handling, logging, reconciliation, and pipeline recovery.
• Ensure every production pipeline includes source‑to‑target mapping, ownership, data quality rules, monitoring, alerting, and operational handover documentation.
• Review vendor and contractor deliverables for technical quality, maintainability, security, performance, and production readiness.

Data Quality, Governance & Production Readiness

• Implement practical data quality controls for completeness, uniqueness, validity, freshness, referential integrity, and reconciliation to source systems.
• Support data governance by ensuring datasets have clear owners, stewards, classifications, lineage, refresh frequency, and required documentation before go‑live.
• Work with business, ERP, and product teams to understand source system meaning, schema changes,…