Senior Data Engineer

Are you an experienced, passionate pioneer in technology who wants to work in a collaborative environment? As an experienced Senior Data Engineer you will share new ideas and collaborate on projects as a consultant without the extensive demands of travel at Deloitte under our Project Delivery Talent Model. The Project Delivery Model (PDM) is tailored specifically for long-term, onsite client service delivery.

• Communicate regularly with Engagement Managers (Directors), project team members, and representatives from various functional and/or technical teams, including escalating any matters that require additional attention and consideration from engagement management
• Design, develop, and optimize ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks
• Write and tune PySpark / Spark SQL notebooks for large-scale data transformation
• Architect end-to-end data solutions across dev > UAT > prod environments using Unity Catalog
• Lead and drive design discussions with client architects and other counterparts
• Collaborate with different teams on data contracts and schema agreements
• Lead design and optimization of high-volume data pipelines
• Define and enforce data engineering standards- naming conventions, partitioning strategies, cluster configurations, Spark tuning
• Drive performance optimization- AQE tuning, liquid clustering, broadcast joins, shuffle partition management
• Design Databricks cluster policies, autoscaling configurations, and cost optimization strategies
• Conduct root-cause analysis on production incidents and implement permanent fixes
• Mentor junior and mid-level engineers through code reviews and pair programming
• Evaluate new technologies and recommend adoption (e.g., DABs, DLT, Auto Loader, Serverless Compute, event hubs)

• Python, PySpark, Spark SQL, SQL Server
• Azure (ADF, ADLS Gen2, Key Vault, Azure Monitor)
• Databricks (Delta Lake, Unity Catalog, Workflows)
• Apache Airflow
• Git / Azure Dev Ops
• Deep Spark internals (DAG optimization, spill analysis, skew handling)
• Delta Lake advanced features (time travel, deletion vectors, predictive I/O)
• Unity Catalog governance (row/column security, external locations, system tables)
• IaC – Terraform, Azure ARM templates
• Bachelor’s degree, preferably in Computer Science, Information Technology, Computer Engineering, or related IT discipline; or equivalent experience
• Limited immigration sponsorship may be available
• Ability to travel 10%, on average, based on the work you do and the clients and industries/sectors you serve

The wage range for this role is $95,000 to $150,000, reflecting factors such as skill set, experience, training, and other business needs. Information for applicants with a need for accommodation is available through Deloitte’s assistance for disabled applicants program.