Sr Data Engineer - Remote

We are seeking a Senior / Lead Data Engineer to join a mature enterprise data platform team built on Databricks, Spark, and DBT
. This role is designed for an engineer who can lead, own, and drive work
, not simply execute tasks.

The environment is past early build‑out and moving into scale, refinement, and optimization
. The engineer will work across ingestion pipelines and refined (gold‑layer) assets
, while also mentoring junior engineers and taking accountability for delivery.

This is not an order‑taker role
. Candidates must bring ideas, make design decisions, and lead work streams independently.

• Lead the design, development, and optimization of data ingestion pipelines using Databricks and Spark
• Build and maintain refined (gold‑layer) data assets using DBT and SQL
• Author and manage DBT models and macros to create reusable, scalable transformations
• Work within a medallion architecture (bronze / silver / gold layers)
• Partner with product owners, engineers, and stakeholders to understand data use cases
• Take ownership of work streams and mentor junior engineers, including task delegation and progress oversight
• Identify performance, scalability, and cost‑optimization opportunities in Databricks workloads
• Contribute to reusable ingestion and transformation frameworks used across teams

Candidates must have strong, hands‑on experience with:

• Databricks (production experience required)
• Apache Spark / Py Spark
• DBT, including DBT macros
• Python
• SQL (used primarily through DBT)
• Strong understanding of data warehousing and medallion architecture concepts

This role requires real implementation depth
, not surface‑level exposure.

• Senior / Lead‑level engineer (title history as Lead is a strong plus)
• Proven ability to:
  • Take ownership and accountability
  • Drive solutions independently
  • Lead and mentor junior engineers
• Comfortable operating in a team of 7–8 engineers
• Able to stay on a project long‑term (1+ year mindset strongly preferred)

• Airflow (primary orchestration tool)
• Experience working with hundreds of pipelines in large enterprise environments
• Prior experience building reusable data assets and frameworks
• Financial services exposure (banking, investments, insurance) — preferred but not required

• Financial services experience is not a hard requirement
• Candidates must be:
  • Willing to learn business context (banking, mutual funds, brokerage systems)
  • Interested in becoming a data SME who can answer stakeholder questions

Ability to connect technical work to business value is a strong differentiator