Principal Data Engineer

Engineers and Constructors International Inc. is hiring a Principal Data Engineer for a contract assignment in Houston, TX (onsite) with an anticipated 7-month duration. In this role, you will lead Azure-based data lake and data governance capabilities, building transformation and streaming pipelines for high-volume data environments.

This Principal Data Engineer contract position focuses on designing and delivering an Azure data platform that leverages Databricks Delta Live Tables
, Structured Streaming
, and Unity Catalog
. You will also help define capability-building roadmaps across additional data technologies and guide technical execution to ensure quality, governance, and production stability.

• Architect, design, and implement scalable Azure-based data lake solutions.
• Establish roadmaps for capability-building across Qlik Replicate
  , Databricks Delta Live Tables
  , and Unity Catalog
  .
• Provide technical leadership to ensure adherence to best practices and quality standards
  .
• Lead development of data transformation pipelines using Databricks Delta Live Tables and Structured Streaming
  .
• Optimize Databricks resources for high-volume processing while staying within budget
  .
• Oversee use of Databricks Unity Catalog for data governance, lineage, and metadata management.
• Develop and enforce robust data governance policies to support compliance, security, and data quality
  .
• Monitor data pipeline performance and proactively resolve production issues.
• Collaborate with cross-functional teams to align project objectives with business goals.
• Translate technical challenges into actionable insights for executive teams
  .
• Mentor and develop a high-performing team skilled in Azure
  , Databricks
  , and data streaming technologies.
• Facilitate knowledge transfer sessions to help internal teams adopt cutting-edge methodologies.

• Bachelor’s or Master’s degree in IT or a related discipline.
• 15-17 years of experience in the IT industry, with expertise in delivering cost-effective cloud-based data analytics solutions.
• Proven experience building data lake architectures and optimizing performance in big data environments.
• Certification in Azure Data Engineering or Databricks (preferred).
• Strong knowledge of Azure technologies including ADF
  , ADLS
  , Databricks
  , Stream Analytics
  , Delta Live Tables
  , and Synapse Analytics
  .
• Advanced proficiency in SQL and programming languages such as Python
  , Scala
  , or Spark-SQL
  .
• Experience designing and developing scalable ETL pipelines and optimizing data workflows.
• Deep understanding of data migration:
  on-premises to cloud and cloud to cloud
  .
• Proven ability to lead diverse teams and manage end-to-end analytics projects
  .
• Exceptional communication and collaboration skills for executives and stakeholders.
• Strong ownership and accountability, with adaptability and a commitment to results.
• Expertise in identifying risks and implementing effective mitigation strategies.

Azure
, Qlik Replicate
, Databricks Delta Live Tables
, Unity Catalog
, Databricks
, Structured Streaming
, SQL
, Python
, Scala
, Spark-SQL
, ADF
, ADLS
, Stream Analytics
, Synapse Analytics
, Hadoop, Kafka,
PowerBI
, Qlik Sense
, AI/ML, NLP, GenAI, AGILE, KANBAN, Mule Soft, Shell IDT/Downstream & Renewables IDT/Trading & Supply/TS
- Crude & Products/TS
- Products/TS
- Products Investments, VIP (Voyage Management System), IMOS, ETRMs, Mule Soft

• Eligibility: You must be a US Citizen or Permanent US Resident to qualify.

The work includes building a new Freight Platform to replace RADAR and serve freight business needs and customers. The platform is expected to act as a hub for seamless integration of data to support an ecosystem of IT tools aligned with Shell’s business growth roadmap. The…