Data Engineer & Architect
Houston, Harris County, Texas, 77024, USA
Listed on 2026-07-28
-
IT/Tech
Data Engineering, Data Warehousing
Cactus Wellhead - Data Engineer & Architect
This is a Cactus Wellhead position and is located in Houston, TX. Looking for local candidate. In office 4 days; work from home 1 day (option).
JOB SUMMARY:
The Senior Data Engineer / Data Architect is responsible for designing, building, and governing the enterprise data platform on Azure Databricks. This role owns both hands-on data engineering delivery and data architecture design, ensuring that enterprise data is reliable, scalable, secure, and ready to support analytics, reporting, and AI initiatives.
The position plays a critical role in transforming data from ERP, CRM, and other enterprise systems into trusted, reusable data products that enable business insights and future AI capabilities. The role operates with high ownership across the full lifecycle, from ingestion and transformation to data modeling, governance, and platform optimization, aligned with a modern Lakehouse architecture.
ESSENTIAL FUNCTIONS,
ROLES AND RESPONSIBILITIES:
Essential duties and responsibilities include the following:
- Design and build end-to-end data pipelines (batch and near real-time) using Databricks and Spark
- Implement and maintain Bronze / Silver / Gold data architecture layers
- Ingest data from ERP, SaaS applications, APIs, and legacy systems into Lakehouse
- Optimize pipelines for performance, scalability, and cost efficiency
- Implement CI/CD pipelines using Git Hub
- Define and maintain the enterprise data architecture and standards
- Design logical and physical data models across core business domains (finance, operations, field service)
- Establish consistent definitions for critical entities (customer, job, revenue, etc.)
- Ensure data structures support both analytics and AI/ML workloads
- Implement and manage Unity Catalog–based governance (RBAC, lineage, auditability)
- Define and enforce data standards for naming, access control, and data quality
- Ensure compliance with security policies (e.g., sensitive HR and financial data protection)
- Establish monitoring, validation, and reconciliation processes to ensure data accuracy and reliability
- Define scalable patterns for:
- Data ingestion
- Transformation pipelines
- Data access (BI tools, APIs, AI models)
- Prepare and structure data for AI/ML use cases and automation initiatives
- Partner with analytics and AI teams to deliver datasets supporting predictive models and AI-driven workflows
- Enable reusable datasets and feature-ready data pipelines
- Partner with business stakeholders, application teams, and analytics teams to translate requirements into scalable solutions
- Provide technical leadership for data engineering best practices and architecture decisions
- Mentor junior engineers or contractors as the platform scales
EDUCATION, TRAINING,
EXPERIENCE:
Experience
- 6–10+ years of experience in data engineering, data architecture, or enterprise data platforms
- Hands-on experience building and supporting data pipelines in cloud environments
- Experience working with enterprise systems (ERP, CRM, or similar)
Technical Skills Strong expertise in:
- Azure Databricks / Apache Spark
- Python / Py Spark
- SQL (advanced)
Experience with:
- Azure Data Lake Storage (ADLS)
- CI/CD with Github
- API and data integration patterns
Strong understanding of:
- Data modeling (dimensional and normalized)
- Data governance and security (RBAC, lineage)
Preferred Qualifications
- Experience with Delta Lake and Lakehouse architecture
- Exposure to ML/AI pipelines or data science workflows
- Experience in manufacturing, oil & gas, or ERP-heavy environments
JOB KNOWLEDGE, SKILLS, ABILITIES:
Familiarity with modern programming languages and frameworks (C#,.NET, SQL, JavaScript). Experience with Dev Ops, CI/CD, and source control for ERP development. Knowledge of cloud-based ERP solutions and migration strategies. Strong background in data integration, ETL, and reporting tools (Power BI, SSRS,…
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).