Skip to content
×

Experienced Director, Lead, Enterprise Data Architecture

Job in New York, New York County, New York, 10261, USA
Listing for: Evercore Inc.
Full Time position Listed on 2026-11-16
Job specializations:
Salary/Wage Range or Industry Benchmark: 210000 - 230000 USD Yearly USD 210000.00 230000.00 YEAR

Job Description & How to Apply Below

Experienced Director, Lead, Enterprise Data Architecture

Region Americas

Location New York

Group Global Technology

What is the Position?
Evercore seeks to hire Director, Lead, Enterprise Data Architecture to serve as the technical authority for data engineering standards and platform architecture within Global Technology’s AI, Data & Analytics organization. This role is responsible for defining and governing how data pipelines, data products, and AI workloads are architected, built, secured, and operated on the firm’s Databricks platform.

This high-impact role sits at the intersection of platform engineering, distributed systems design, and developer experience. The ideal candidate will bring deep expertise in Databricks, Spark, enterprise data governance and event-driven architectures for regulated environments.

Primary Responsibilities:
We are seeking an experienced technical leader with deep Databricks expertise to define strategy and establish architectural standards across the platform. This role will partner closely with business, technology, security, and infrastructure leaders to ensure the platform is scalable, compliant, cost-efficient, and aligned to long-term enterprise objectives. Responsibilities include, but are not limited to the following:

  • Define and govern enterprise architecture standards for batch and streaming data pipelines on Azure Databricks, including Spark optimization, structured streaming, and pub/sub integration (e.g., Event Hub, Kafka).
  • Design and maintain standardized internal SDKs and reusable frameworks for ETL development.
  • Establish the data engineering SDLC, including schema evolution policies, artifact versioning, environment promotion controls and CI/CD standards.
  • Architect and administer secure, scalable Databricks environments, including workspace segmentation strategies, Unity Catalog design, role-based and attribute-based access controls, row/column-level security, and compute isolation policies.
  • Design and implement enterprise metadata, observability, and data quality frameworks, including automated lineage capture, SLA monitoring, operational telemetry, and cost attribution models.
  • Establish data product standards, including schema contract enforcement, versioning policies, backward compatibility requirements, domain ownership models, and publish/subscribe consumption patterns.
  • Define best practices for exploratory and non-production workloads, including guardrails for data scientist experimentation, compute policy controls, secure data access boundaries, and promotion of validated assets into production pipelines.
  • Partner with Data Scientists and Machine Learning Engineers to define the lifecycle from experimentation to production, including feature engineering standards, model registry governance, reproducibility controls, and approved deployment pathways.


Specific

Qualifications:

  • Minimum 10+ years of experience in data engineering, enterprise data platform architecture, or large-scale analytics platforms.
  • Minimum 5+ years of hands-on experience designing and scaling Databricks and Spark-based platforms in enterprise environments.
  • Deep expertise in Apache Spark (performance tuning, optimization strategies, structured streaming, distributed compute patterns).
  • Experience architecting Azure cloud infrastructure.
  • Proven experience architecting event-driven and streaming data systems using pub/sub technologies such as Kafka or equivalent platforms.
  • Demonstrated experience designing enterprise-grade ETL/ELT frameworks and internal SDKs for standardized pipeline development and deployment.
  • Strong experience implementing Unity Catalog or equivalent enterprise data governance tooling, including fine-grained access control models and regulatory audit readiness.
  • Experience…
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction

Search for further Jobs Here:

SEARCH JOBS: 10+ Million Listings
(Try combinations for better Results! Or enter less keywords for broader Results)
Education Level
Experience Level (years)
Posted in last:
Salary