The Role
We're engaging a hands‑on / Senior Databricks platform engineer/builder for our client - one of global leading strategy‑led tech consultancy company. The objective of role is to lead the design and operationalisation of a governed, multi‑tenant data platform on Azure Databricks - ingesting and onboarding data from multi external organisations across Databricks, Snowflake, API, and file‑based sources.
Design and implement governance from the ground up, and make Unity Catalog and Delta Sharing work at scale in production, contributing to the low level design of the architecture already selected.
What You’ll Be Doing- Unity Catalog governance — design and implement the catalog/schema structure, privilege model, data ownership, and lineage for a platform serving multiple business units and external partners
- Delta Sharing — architect the provider/recipient model for external data onboarding; manage refresh strategy, access scoping, and cross‑platform sharing (Databricks-to-Databricks, Databricks-to-Snowflake)
- Identity & access integration — Microsoft Entra n with Unity Catalog; service principal design; RBAC across environment tiers
- Pipeline modernisation — refactor existing Python pipelines into a production‑grade Databricks medallion architecture (bronze/silver/gold)
- Platform engineering — workspace structure, environment promotion, CI/CD, cost optimisation, and capacity planning
- Architecture contribution — this is a live engagement with an existing recommendation in place; we want someone who will challenge and improve it, not just implement it
Must‑Have
- Hands‑on Unity Catalog implementation in a production environment — not workshop‑level exposure. You should be able to talk through catalog structure, privilege model, and lineage from real deployments.
- Real Delta Sharing experience — provider/recipient architecture, external data onboarding, operational challenges you’ve actually hit and solved
- Strong Python and Py Spark , with production pipeline migration experience (not just greenfield builds)
- Microsoft Entra integration experience — service principals, RBAC, scoped access
- Track record of making platform architecture decisions yourself, not just executing someone else’s design
- 7+ years data platform engineering, 3+ years hands‑on Azure Databricks
- Databricks Genie evaluation/implementation experience
- Azure Data Factory orchestration
- CI/CD and Data Ops (Azure Dev Ops, Databricks Asset Bundles)
- Fin Ops / platform cost optimisation
- Regulated industry or enterprise governance experience
If you’re an experienced builder in applying Unity Catalog & Delta Sharing to external Data integration, and looking to work alongside a top‑tier global consultancy on a high visibility platform build / product‑grade delivery, we’d like to hear from you.
#J-18808-Ljbffr(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).