Data Engineer: Lakehouse Automation & AWS Data Platform
Listed on 2026-10-31
-
IT/Tech
Data Engineering, AWS, Cloud Computing: Infrastructure & Operations
Job Description
:
Automate creation, transformation, and integration of raw and enriched data models in the data Lakehouse.
Implement automated ETL pipelines for data ingestion, transformation, and consumption
Design and implementation for integrating Lake formation across AWS Data Platform Accounts.
Design and implementation for integrating Lake formation with Keycloak.
Design and implementation for an integrated IAM solution including EKS Service Accounts.
Design and implementation of Hashi Corp Vault with integration to AWS Services such as Secrets Manager, Certificate Manager and Keycloak.
Development of integration of Hashi Corp Vault with AWS Data Services such as RDS Aurora PostgreSQL, RDS MSSQL and other RDS solutions as required.
Provision MSSQL server on AWS using Terraform
Deploy read replica for MSSQL server using Terraform
Implement monitoring & logging for MSSQL server using Terraform
Transition existing pipeline to MSSQL server
Collaborate with the business application owner on the existing data architecture, including data ingestion, data pipelines, business logic, data consumption patterns, and analytics requirements
Design and document the target data architecture, pipelines, processing and analytics architecture
Identify opportunities for optimization and consolidation
Collaboration with data team on decomposition of business logic and data transformation patterns
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).