Databricks Data Solution Architect
Listed on 2026-09-30
-
IT/Tech
Data Engineering
We partner with global enterprises to design and build large-scale data platforms that power the products and operations at the center of their business. This engagement is a Databricks lakehouse foundation for a US defense manufacturer, which is establishing Databricks as the central platform for their engineering and manufacturing data.
Most of that data is export-controlled. Access cannot be a per-table exercise here; it has to derive from classification, which means governance is not a layer added later: it is the architecture. You will build the classification model, express it as a Unity Catalog policy, and defend it to enterprise security reviewers.
This is a technical leadership role. You will lead through architecture, technical credibility, and influence, with one senior engineer on the team alongside you. Two people on the engagement means you own the architecture and governance end to end, run stakeholder discovery yourself, and present without an account team in between.
The engagement opens with a discovery and architecture phase: assess the current landscape, prioritize use cases, and author the target-state design, governance model, and implementation roadmap. It then moves into the build, executing what you wrote.
What You’ll Do- Lead discovery across the client's data, engineering, security, and business teams, documenting data movement, security architecture, integration topology, and platform configuration, and setting the cadence, RACI, and communication plan.
- Prioritize the work by facilitating use case sessions and scoring candidates on business value, technical complexity, and data readiness.
- Design the target-state Lakehouse architecture, including medallion design across structured relational sources, Delta Lake patterns (managed versus external tables, predictive optimization, liquid clustering), orchestration in Databricks Workflows, and federate-versus-ingest decisions per source using Lakehouse Federation.
- Own the Unity Catalog governance blueprint: metastore topology, row filters, column masks, attribute-based access control, governed tags, and lineage, with CUI and ITAR classification expressed as tag-driven policy so access derives from classification rather than per-table maintenance.
- Apply Fin Ops discipline through budget and cluster policies, serverless spend management, and chargeback tagging, alongside data quality monitoring standards and the semantic layer.
- Present to enterprise security reviewers and carry the architecture through their process.
- Author the deliverables: current-state assessment, target-state architecture, implementation roadmap, and the executive readout to client leadership.
- Lead the build, implementing the architecture on AWS, including Gov Cloud deployment where required, workspace topology, network isolation, identity integration, Unity Catalog governance, and Delta Sharing across regulatory boundaries.
- Set technical direction for the ingestion engineer, review their work, and unblock them on source system and platform issues.
- 8+ years across solutions architecture, data engineering, or related roles, including multiple production Databricks platform architectures on AWS Gov Cloud. At least one should be in a regulated or export-controlled environment.
- Deep, hands-on Unity Catalog expertise. You have implemented row filters, column masks, and attribute-based access control in production and can explain the trade-offs of each.
- Practical experience translating regulatory requirements (CUI, ITAR, or equivalent classification regimes) into enforceable data policy.
- Working knowledge of Delta Lake internals, Lakehouse Federation, Delta Sharing, and Databricks Workflows sufficient to make and defend design decisions under…
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).