Senior Data Lakehouse Architect; Databricks), Vice President

About the Role

We are seeking a Senior Data Lakehouse Architect to design and lead the build-out of a Legal Data Lakehouse platform on AWS and Databricks
. This role will drive the architecture, engineering, and governance of scalable, secure, and compliant data capabilities supporting legal operations, contract intelligence, eDiscovery, and AI/ML use cases.

State Street’s Legal function operates across a wide range of contracts, matters, regulatory obligations, documents, and workflows. Building a modern Legal Data Lakehouse is critical to creating a trusted, governed foundation that brings these data sources together—making legal information easier to access, analyze, and use at scale.

• Architecture & Platform Design:
  Define and implement the end‑to‑end Legal Data Lakehouse architecture using Databricks (Delta Lake, Unity Catalog, Workflows) on AWS.
• Design multi‑layered data architecture (Bronze, Silver, Gold) to support contract metadata and document ingestion, legal matter management data, eDiscovery datasets, and external regulatory and compliance feeds.
• Establish scalable ingestion frameworks (batch and streaming) for structured and unstructured legal data (PDFs, contracts, emails).
• Data Engineering & Integration:
  Lead development of ETL/ELT pipelines using Databricks, Spark, and Python/SQL; integrate with enterprise platforms including contract lifecycle management systems, AI platforms, LLM pipelines, and document repositories.
• Design patterns for extracting structured data from unstructured legal documents and persisting into Delta Lake; enable downstream integration with enterprise data platforms, analytics tools, and AI/ML pipelines.
• Governance, Security & Compliance:
  Implement data governance frameworks using Databricks Unity Catalog and AWS‑native controls (IAM, KMS); establish fine‑grained access controls, data lineage and auditability; ensure compliance with data privacy regulations (e.g., GDPR) and internal security and audit requirements; partner with IAM teams to integrate with enterprise identity providers (Entra  / Azure AD).
• AI/ML & Advanced Analytics Enablement:
  Architect data models supporting contract analytics, clause extraction, and obligation tracking; legal AI use cases (contract review, litigation insights, compliance monitoring, legal spend analytics); design search and retrieval architectures (RAG) for enterprise legal knowledge bases; enable entity extraction and knowledge graph frameworks; integrate with LLM/GenAI platforms for document summarization, Q&A, and workflow automation.
• Dev Ops & Platform Operations:
  Establish CI/CD pipelines and infrastructure‑as‑code (Terraform, Git‑based workflows); define standards for code quality and versioning, environment promotion (Dev / QA / Prod); implement observability and alerting for platform health and reliability.
• Leadership & Stakeholder Engagement:
  Partner with Legal and Technology leadership to define platform roadmap and priorities; provide architectural governance and design oversight; mentor data engineers and platform teams; translate business and legal requirements into scalable, enterprise‑grade solutions; operate within a federated data and platform model, collaborating across engineering, security, and domain teams.

• 10+ years of experience in data architecture, engineering, or analytics platforms.
• 5+ years of hands‑on experience with Databricks and Apache Spark.
• Strong experience with AWS‑based data platforms.
• Expertise in data governance, security, and compliance in regulated environments.
• Experience working with unstructured data and NLP/document processing pipelines.

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related discipline.
• Relevant certifications strongly preferred:
  Databricks Certified Data Engineer / Architect; AWS Certified Solutions Architect (Associate or Professional).

• Strong hands‑on experience with the Databricks Lakehouse platform, including Delta Lake, Unity Catalog, Workflows, and MLflow.
• Deep expertise in AWS data platform services (S3,…