Azure Databricks Architect - Remote with Travel to NJ

• Build and maintain ingestion frameworks (ADF / Databricks / Spark)
• Implement Bronze ->
  Silver transformations aligned to architecture
• Architect data quality checks, schema validation, and contract rules
• Optimize pipelines for performance and cost

• Own CI/CD pipelines (Azure Dev Ops)
• Automate onboarding (repos, pipelines, policies, templates)
• Maintain IaC and scripting (Terraform, Power Shell / Python)
• Troubleshoot pipeline failures and permission issues

• Dimensional models (star/snowflake schemas)
• Fact tables, dimensions, surrogate keys, SCD handling

• Analytics-ready models
• Consistent KPI definitions and business logic

• Consistency across domains (common dimensions, conformed models)
• Reusability and scalability of models
• Alignment with enterprise semantic layer (Power BI / Fabric / downstream tools)

• Column derivation logic changes (not just schema changes)
• Backward compatibility and impact analysis

• Define business rules
• Validate metrics and edge cases
• Support regression scenarios

• Unity Catalog standards
• naming conventions, metadata policies
• access controls (RBAC/ABAC)

• low-level designs
• reusable templates
• engineering playbooks

• Support delivery teams with:
• onboarding
• debugging
• design reviews
• Provide hands-on guidance
• Create reusable assets to reduce repeat questions

• Azure Data Platform (ADF, ADLS, Azure SQL)
• Databricks (DLT, Delta Lake, Spark, Workflows, Unity Catalog)
• Microsoft Fabric (preferred)

Engineering

• PySpark / Python (advanced)
• SQL (strong)
• Pipeline design (batch + streaming)

Dev Ops / Automation

• Azure Dev Ops (pipelines, repos, policies
• Power Shell automation (must-have for your setup)
• CI/CD + Infrastructure-as-Code concepts

Governance / Data Quality

• Schema enforcement, data contracts, DQ frameworks
• Metadata-driven design understanding