Associate Principal - Cloud Engineering

Job Title:

DATABRICKS Architect Work Location Irving TX

Job Description:

Role Overview We are seeking a highly skilled Databricks Engineer with AWS expertise to support Data Services portfolio

The role will be responsible for designing developing optimizing and supporting enterprise scale data platforms data pipelines and analytics solutions leveraging Databricks Lakehouse AWS Cloud and modern data engineering practices The candidate will work closely with data architects’ application teams business stakeholders and platform engineering teams to deliver scalable secure and high-performance data solutions

Key Responsibilities Design develop and maintain scalable data pipelines using Databricks PySpark Spark SQL and Delta Lake Build and support enterprise data lake and Lakehouse solutions on AWS Develop batch and Realtime ingestion frameworks from multiple source systems

Implement data transformation enrichment cleansing and validation processes

Optimize Spark workloads for performance scalability and cost efficiency

Design and implement data models supporting reporting analytics and downstream applications

Integrate Databricks with AWS services including S3 Glue Lambda Redshift Event Bridge and IAMSupport data migration and modernization initiatives from traditional data warehouses to Databricks Lakehouse Implement CICD pipelines and Infrastructure as Code IaC using Terraform and Gitbased repositories

Collaborate with security teams to ensure compliance with client governance risk and data security standards

Troubleshoot production incidents and provide L3 operational support

Participate in architecture reviews code reviews and technical governance activities

Mandatory Technical Skills Databricks Databricks Lakehouse Architecture Databricks Workflows Delta Lake Unity Catalog Databricks SQLAuto Loader Structured Streaming Databricks Performance Tuning Medallion ArchitectureAWSAmazon S3

AWS GlueAWS LambdaAWS IAMAmazon RedshiftAWS Cloud

WatchSNS SQSEvent

BridgeKMSSecrets Manager Data Engineering Py Spark Apache  Spark

PythonSQLData Warehousing ConceptsETLELT Design Data Modeling Data Quality Frameworks Batch Real Time Processing Preferred Skills Snowflake Microsoft FabricGCP Big Query Kafka Airflowdbt Terraform Docker Kubernetes  Open Shift Informatica Collibra Data  Vault Modeling Dimensional Data Modeling Banking Domain Experience Strong experience within Banking and Financial Services Understanding of enterprise data governance regulatory reporting risk compliance and audit requirements

Experience working in large global organizations with stringent security and operational standards Qualifications Bachelors or Masters degree in Computer Science Engineering Information Systems or related discipline

Databricks Certified Data Engineer Preferred AWS Certified Data Analytics or AWS Solution Architect Preferred