Databricks Platform Engineer

We're looking for an Azure Databricks Platform Engineer to take ownership of the administration, governance, security, and operational health of an enterprise Databricks environment. This individual will ensure the platform remains secure, reliable, properly governed, and cost-efficient while supporting a growing portfolio of enterprise data, analytics, and AI workloads.

This is a platform administration role rather than a traditional data engineering position. The ideal candidate has hands‑on experience owning Databricks environments and understands how to manage Unity Catalog, access controls, work spaces, compute resources, production jobs, and platform governance.

• Administer Databricks environments supporting development, testing, pre‑production, and production workloads.
• Own day‑to‑day platform health by monitoring workloads, scheduled jobs, compute resources, availability, and other operational activity.
• Administer Unity Catalog and maintain scalable governance standards for enterprise data, permissions, and access.
• Manage users, groups, roles, and access controls while ensuring sensitive information is appropriately protected across the platform.
• Troubleshoot Databricks platform issues involving permissions, data access, jobs, compute, workspace configuration, and connectivity.
• Configure and optimize clusters and compute resources based on workload requirements, performance, utilization, and cloud cost considerations.
• Establish consistent administrative and security standards as additional users, datasets, and business workloads are introduced to the platform.
• Support controlled promotion of platform configurations and security standards across development and production environments.

• Hands‑on experience administering Databricks in a production enterprise environment
  , with direct responsibility for the platform rather than solely developing notebooks or data pipelines.
• Strong experience with Unity Catalog
  , including governance, permissions, catalogs, schemas, users/groups, and enterprise data‑access controls.
• Experience administering Databricks work spaces, clusters/compute, jobs, security, permissions, and platform configurations.
• Demonstrated experience troubleshooting production Databricks issues and supporting the operational health of the platform.
• Understanding of role‑based access, least‑privilege security, sensitive‑data protection, and scalable enterprise governance practices.
• Experience monitoring and optimizing Databricks compute for performance, utilization, reliability, and cost.
• Experience supporting multiple Databricks environments and maintaining consistent configurations and security standards between them.
• Azure Databricks experience is strongly preferred; candidates with significant Databricks administration experience in another major cloud environment may also be considered.
• Familiarity with Azure identity, storage, networking, monitoring, or security services is preferred.
• Exposure to automation, Infrastructure as Code, Terraform, CI/CD, Git Hub, or Azure Dev Ops is beneficial.
• Knowledge of Microsoft Purview, Microsoft Fabric, Delta Lake, Lakehouse architecture, Power BI, or enterprise AI integrations is a plus.

Join us at Lumen Resources, where your expertise will contribute to our growth, and you'll have the opportunity to have fun and thrive alongside a dynamic group of professionals.

Lumen Resources is an Equal Opportunity Employer. Don't hesitate to get in touch with our team for more information.