Data Engineer – Data Lake

The Data Engineer – Data Lake is responsible for designing, building, and maintaining scalable data pipelines and a modern cloud data lake architecture. The role focuses on ingestion of structured and unstructured data, ETL/ELT development and implementation of the Bronze, Silver, and Gold data layers to enable analytics, reporting, and advanced data use cases across transportation, fleet management, leasing, and maintenance operations.

The ideal candidate brings strong hands-on experience with Azure data services, data modelling, and pipeline orchestration, with a solid understanding of data quality, performance optimization, and analytics-ready data architectures.

• Design and implement a scalable Azure-based data lake architecture following the Bronze, Silver, and Gold layered approach.
• Define data ingestion, transformation, and consumption patterns to support BI, analytics, and data science use cases.
• Collaborate with data architects to align data lake design with enterprise data and integration architecture.
• Ensure data lake structures support both batch and near-real-time data processing where required.

• Configure and manage data ingestion pipelines to ingest data from operational systems, SaaS platforms, APIs, and databases into the data lake.
• Monitor ingestion jobs, manage schema changes, and resolve data sync and quality issues.
• Implement ELT patterns to land raw data in the Bronze layer with full traceability.
• Develop and maintain robust ETL/ELT pipelines to ingest data from multiple sources, including operational systems, IoT platforms, third-party systems, and APIs.
• Implement data transformations, standardization, and enrichment processes across data layers.
• Optimize ELT patterns to leverage cloud-native processing engines (e.g., Spark, SQL pools).
• Ensure pipelines are modular, reusable, well-documented, and scalable.

• Implement data validation, reconciliation, and quality checks across all data layers.
• Support data governance initiatives by applying metadata management, data lineage, and data classification standards.
• Monitor pipeline performance, data freshness, and reliability; troubleshoot and resolve data issues proactively.
• Work closely with data governance and analytics teams to ensure trusted, analytics-ready datasets.

• Integrate the data lake with downstream analytics platforms, BI tools, and data science environments.
• Design and maintain curated Gold layer datasets optimized for reporting, dashboards, and advanced analytics.
• Support data scientists and analysts by ensuring timely access to high-quality, well-structured data.
• Contribute to defining data models and semantic layers aligned with business use cases.

• Implement monitoring, logging, and alerting for data pipelines and platform components.
• Apply security best practices, including access control, data encryption, and compliance with organizational policies.
• Continuously identify opportunities to improve performance, cost efficiency, and reliability of data pipelines.
• Maintain technical documentation for pipelines, data models, and architectural decisions.

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related technical field.
• Master’s degree in a related field is a plus.
• Minimum of 6–8 years of hands-on experience in data engineering, ETL/ELT development, and data platform implementation.
• Strong experience with Microsoft Azure data services, DBT, Five Tran etc
• Proven experience implementing Bronze, Silver, and Gold data lake architectures.
• Strong proficiency in SQL and experience with PySpark / Python for data transformations.
• Experience integrating data from operational systems, APIs, IoT platforms, and third-party data sources.
• Solid understanding of data modeling concepts (dimensional, analytical, and curated data models).
• Familiarity with data quality frameworks, metadata management, and data governance principles.
• Experience working in analytics-driven environments supporting BI, reporting, and data science teams.
• Strong problem-solving skills and ability to work with both technical and business stakeholders.
• Domain experience in transportation, fleet management, leasing, or maintenance is a strong asset.