Principal Data Architect - Databricks & AI

The opportunity

We are investing in the next generation of our data and technology capabilities to create more connected, intelligent experiences for customers. Data is central to this ambition. This is an opportunity to define the architecture, standards and delivery patterns for a modern data platform that will enable trusted data use across products and support advanced analytics and AI.

The Principal Data Architect will be the senior technical authority for data architecture within this strategic transformation. The role is architecture-led while remaining practically engaged through prototyping, proof-of-concepts, technical reviews and the resolution of complex engineering challenges. As the capability grows, the remit may expand to include leadership of a small team.

• Define and own the target data architecture, standards, reference patterns and technical roadmap for a strategic enterprise data capability.
• Design a modern Databricks lakehouse foundation that enables secure, scalable use of data across a complex product landscape.
• Establish scalable medallion patterns across
  
  Bronze, Silver and Gold layers, including ingestion, transformation, storage,serving and consumption.
• Design canonical,dimensional and semantic data models that enable consistent exchange,interoperability and reuse across product domains.
• Architect secure data integration using batch, streaming, APIs, change datacaptureand event-driven patterns.
• Design privacy-preserving data capabilities, including data masking,anonymisationor equivalent controls for customer data used in AI and analytical workloads.
• Design AI-ready data capabilities, including governed model-data pipelines, vector search and retrieval-augmented generation patterns where relevant.
• Create prototypes and proof-of-concepts, review technical designs and code, guide performanceoptimisation, and help resolve complex technical issues.
• Build production-ready solutions with strong governance, lineage, quality, observability, security,reliability and cost control.
• Partner with product, engineering, AI, security,platform and business leaders to translate customer and product needs into architecture and delivery plans.
• Mentor data engineers and architects and promote reusable platform capabilities and engineering standards.

• Significant experience as a Data Architect, Lead Data Architect, Principal Data Engineeror comparable senior technical leader.
• Advanced, hands-on Databricks experience in production environments, includinglakehousearchitecture, Delta Lake and relevant governance, workflow,SQLandoptimisationcapabilities.
• Proven ownership of modern data platforms or data products from architecture through production delivery and operation.
• Strong experience in data privacy and security, including data masking,anonymisation,tokenisationor comparable privacy-preserving patterns for sensitive or customer data.
• Strong data-modellingexpertiseacross conceptual, logical, physical,dimensional and canonical models.
• Experience with batch and real-time integration, including APIs, CDC,streamingor event-driven pipelines.
• A strong data-engineering or software-development foundation using Python, SQL, Scala,Javaand/or Spark.
• Deep experience with Azure data services. AWS experience isadvantageousas the platform expands.
• Experience with governance, cataloguing, lineage, quality, metadata, access control and secure multi-tenant or customer-data environments.
• Credibility with hands-on engineers and senior stakeholders, with strong judgement across speed, scale, governance,costand usability.

• Production AI/ML or generative-AI data architecture, including RAG, embeddings, vectorsearchormodel-data pipelines.
• Knowledge graphs, ontologies, RDF,…