Data Governance - Unity Catalog

Purple Drive

Data Governance
- Unity Catalog Share Seattle, WA

Overview:
Your Role

Implement data provisioning patterns based on business requirements, following predefined processes, policies, standards, and metadata management rules

Create and manage distributed work spaces in Databricks, set up workspace policies, provision Databricks clusters and manage data infrastructure sizing and capacity

Create Python notebooks, implement data masking processes, create UDFs (SQL/Python), troubleshoot data pipelines

Ensure data security and compliance with regulations using Databricks and Privacera's features

Navigate multi-step enterprise approval process across architecture, security, and governance teams

Design and implement data architecture leveraging technologies such as Databricks, Unity Catalog, and Privacera.

Develop, optimize, and manage data pipelines for ETL processes using Databricks, with a focus on data integrity and quality

Design and maintain data models and schemas, incorporating Unity Catalog and Collibra data governance practices

Establish a robust data governance strategy, defining standards, metadata management, lineage, and quality practices

Operationalize Machine Learning models in Batch and Real Time Data Pipelines, leveraging relevant governance setups

Collaborate with cross-functional teams including data scientists, engineers, and analysts to translate business requirements into scalable solutions

Foster collaboration and clarity in complex, ambiguous environments.

Job Description
- Grade Specific

Your skills and experience:

Extensive hands-on platform engineering experience with Databricks and Unity Catalog

Proven success in implementing distributed Databricks work spaces, SQL, and Python, scripting, and automation

Experience with Azure infrastructure provisioning

Familiarity with data governance frameworks and compliance standards

Familiarity with Privacera data security and access control management

Familiarity with data requirements of common ML/AI use cases

Awareness of data governance frameworks, enterprise data compliance requirements, metadata modeling, data architecture, and enterprise-scale data discovery solutions