Principal Data Platform Engineer

Exactera has offices in New York City, Tarrytown NY, San Diego, CA, London, and Argentina.

As Principal Data Platform Engineer, you'll architect and implement our centralized data platform on Databricks. You'll establish governance patterns using Unity Catalog, optimize for cost and performance at scale, and enable our existing Data Engineers to build confidently on the platform. This is a data infrastructure role—focused on pipelines, storage, governance, and platform operations.

We operate multiple product lines (Transfer Pricing, R&D Services, Royalty Stat, Provisioning), each with distinct databases containing enterprise financial data—journal entries, general ledgers, and financial statements. Our immediate challenge is migrating multi‑terabyte datasets from legacy systems to a unified Databricks lakehouse while establishing governance patterns that enable multi‑product operations at scale.

• Design data models and implement unified schemas across multiple disparate product lines.
• Design and implement a multi‑catalog governance strategy supporting data isolation, cross‑product data sharing, and comprehensive lineage tracking across the portfolio.
• Establish patterns for Z‑ordering, compaction, and liquid clustering at multi‑terabyte scale. Define table structures, partitioning strategies, and retention policies that balance query performance with storage costs.
• Build declarative pipeline patterns using Delta Live Tables. Create orchestration workflows for ingesting data from internal sources such as SQL databases and S3.
• Integrate with third‑party data sources such as ERP systems (Net Suite, etc.) and external data providers (S&P, etc.) with automated ingest, robust error handling, and monitoring.
• Implement cost monitoring and optimization strategies, establish data quality frameworks, and create self‑service patterns enabling Data Engineers to work independently while maintaining governance standards.

• Lead the architecture for migrating multi‑terabyte datasets from legacy systems to Databricks—establishing reusable patterns across product lines.
• Design Unity Catalog structures enabling secure data separation between product lines while allowing controlled cross‑product analytics where appropriate.
• Build infrastructure that scales efficiently—through intelligent caching, query optimization, and compute management strategies that avoid linear cost growth.
• Establish monitoring, alerting, and data quality validation ensuring the platform operates reliably as a foundation for both analytics and AI workloads.

• Production experience with multi‑catalog governance, metastore design, and lineage tracking.
• Designing and building unified schemas across multiple disparate product lines.
• Expert‑level experience with Z‑ordering, compaction, liquid clustering, and performance tuning at multi‑terabyte scale.
• Hands‑on experience building declarative ETL pipelines with Delta Live Tables, including change data capture and expectations/constraints.
• Experience with Databricks workflows, job orchestration, scheduling, and operational monitoring.
• Experience enabling company‑wide analytics and reporting with modern business intelligence tools, maintaining source‑of‑truth data and metrics.
• Strong proficiency in PySpark and Databricks SQL for code review, performance tuning, and query optimization.

• 5–8 years in data engineering or data platform roles, with 3+ years hands‑on Databricks experience.
• Track record leading at least one significant platform build or migration project.
• AWS experience (S3, IAM, VPC) with ability to collaborate on infrastructure decisions.
• Infrastructure‑as‑code experience (Terraform preferred).

• Demonstrated ability to architect data platforms from first principles and defend technical decisions.
• Strong written and verbal communication—document architecture decisions and present to both technical and business stakeholders.

• Experience with financial data, accounting systems (Net Suite), or enterprise ERP platforms.
• Background building platforms that…