Senior Databricks Specialist​/Data Architect

Position Summary

We are seeking an exceptionally skilled and experienced Senior Databricks Specialist / Data Architect to join Marc Ellis on a 12-month extendable contract basis (long-term under Marc Ellis payroll) in Doha, Qatar. In this critical enterprise data leadership role, you will be responsible for designing, implementing, and optimizing enterprise-scale data platforms using Microsoft Azure Databricks, Apache Spark, and Delta Lake architectures.

The ideal candidate will bring 8+ years of experience in data engineering and architecture, at least 4+ years of strong hands-on Azure Databricks expertise, and deep technical familiarity with Unity Catalog and Medallion Architecture. If you possess a distinguished background in big data platform engineering and multi-system integration, we invite you to apply.

As a Senior Databricks Specialist / Data Architect at Marc Ellis in Doha, Qatar, you will serve as the principal technical authority responsible for architecting, governing, and scaling the organization’s enterprise data platform and its integration with Dynamics 365 CRM and other core enterprise systems. Your daily responsibilities will encompass designing robust data pipelines using Azure Data Factory, implementing Delta Lake storage layers, configuring Unity Catalog governance and access controls, and driving performance tuning and cost optimization on Azure Databricks.

You will troubleshoot complex production data issues and collaborate with engineering teams to automate CI/CD deployments via Azure Dev Ops.

• Design, implement, architect, and optimize enterprise-scale data platforms using Microsoft Azure Databricks.
• Build and scale high-performance data pipelines and processing frameworks using Apache Spark and PySpark.
• Architect and manage Delta Lake storage layers and implement Databricks Medallion Architecture (Bronze, Silver, Gold).
• Implement enterprise data governance, access control, and data lineage tracking using Unity Catalog.
• Integrate Azure Databricks seamlessly with APIs, enterprise applications, Dynamics 365 CRM, and auxiliary data sources.
• Design and maintain robust cloud data infrastructure using Azure Data Factory and Azure Data Lake Storage Gen
  2.
• Apply advanced data modeling, dimensional modeling, and data quality standards across large enterprise datasets.
• Drive continuous performance optimization, cost reduction, and query tuning on Databricks clusters.
• Build automated CI/CD deployment pipelines, version control workflows, and infrastructure as code using Azure Dev Ops and Git.
• Troubleshoot complex production data issues, memory bottlenecks, and pipeline failures rapidly.

• Bachelor's degree in Computer Science, Data Science, Information Technology, Software Engineering, or a related discipline.
• Minimum of 8+ years of professional experience in data engineering, data architecture, or big data systems.
• Minimum of 4+ years of strong hands-on professional experience with Microsoft Azure Databricks.
• Deep technical proficiency in Apache Spark, PySpark, and Delta Lake data formats.
• Strong experience designing enterprise data platforms, data pipelines, and Databricks Medallion Architectures.
• Advanced practical knowledge of Unity Catalog, data governance, security policies, and access controls.
• Strong SQL skills and proven capability in processing large-volume enterprise datasets.
• Solid background with Azure data services including Azure Data Factory and Azure Data Lake Storage Gen
  2.
• Experience with CI/CD deployment automation, Git, and Azure Dev Ops pipelines.
• Mandatory residency/work authorization:
  Ability to operate in Doha, Qatar.

• Professional certifications such as Databricks…