Senior Azure Databricks Engineer

Get AI-powered advice on this job and more exclusive features.

Fractal Analytics is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. Fractal is building a world where individual choices, freedom, and diversity are the greatest assets. An ecosystem where human imagination is at the heart of every decision. Where no possibility is written off, only challenged to get better. We believe that a true Fractalite empowers imagination with intelligence.

And that it will be such Fractalites that will continue to build the company for the next 100 years.

Please visit Fractal | Intelligence for Imagination for more information about Fractal.

We are seeking a skilled Lead Data Engineer to support the migration of on-premise SQL Server databases to the Azure Databricks Lakehouse Platform. This role involves designing and implementing parameter-driven ETL frameworks, migrating SQL stored procedures, views, and business logic to Databricks Unity Catalog stored procedures, and supporting data governance strategies. The ideal candidate will have proven experience with Databricks, Unity Catalog, Delta Lake, and ADF, and will work closely with the team to deliver scalable, efficient, and secure data migration solutions.

• Design and implement data migration pipelines from on-prem SQL Server to Azure Databricks Lakehouse.
• Build parameter-driven/config-based ETL mappings for reusable and dynamic pipelines.
• Migrate SQL stored procedures, views, and business logic from SQL Server to Databricks Unity Catalog stored procedures within the Databricks Lakehouse using Databricks SQL or PySpark.
• Configure and manage Databricks Unity Catalog for governance, lineage, access control, and security.
• Develop and optimize Delta Lake tables following the Medallion architecture (Bronze, Silver, Gold).
• Orchestrate ETL workflows via Azure Data Factory (ADF) and Databricks Workflows.
• Implement data access controls, account-level permissions, and object ownership via Unity Catalog.
• Collaborate with architects, developers, and business stakeholders to ensure data quality, lineage, and governance.
• Troubleshoot and optimize ETL pipelines for performance, scalability, and cost efficiency.
• Support CI/CD integration using Databricks Repos, Git, and Azure Dev Ops.

• 10+ years of experience in data engineering, including cloud migration projects.
• Hands-on expertise in Azure Databricks, ADF, ADLS Gen2, and Databricks Unity Catalog.
• Strong knowledge of SQL Server stored procedures, SSIS, SSRS.
• Proven experience in migrating SQL stored procedures, views, and business logic to Databricks Unity Catalog stored procedures.
• Strong skills in Databricks SQL, PySpark, and Python for data transformation.
• Deep understanding of Delta Lake, Medallion architecture, and data modeling.
• Knowledge of data governance, lineage tracking, and security policies.
• Experience with CI/CD automation, Git, and Azure Dev Ops.

• Current pay range: $120,000 - $145,000. Compensation decisions are based on various factors including skills, experience, training, licensure, and business needs. A discretionary bonus may apply.
• Full-time or eligible hourly employees: health, dental, vision, life insurance, and disability plans. Eligible to participate in the Company 401(k) Plan after 30 days. 11 paid holidays and 12 weeks of Parental Leave. Free time PTO policy for sick time or vacation.

Fractal provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

• Seniority level:
  Mid-Senior level
• Employment type:
  
  Full-time
• Job function:
  Information Technology
• Industries:
  Business Consulting and Services

Refer

Apply BELOW