Data Engineering Lead

Builders is committed to reliability, fairness, and customer-first service, earning an A rating by AM Best and recognition through Top Workplaces “Culture Excellence” honors. This role supports an insurance business built on strong partnerships and mutual respect, with an onsite opportunity in Atlanta, GA to lead a platform that helps teams modernize and govern data for property and casualty insurance.

As the technical owner of the Azure Databricks lakehouse platform, you will drive architecture, modernization, governance alignment, and engineering leadership to deliver a scalable, secure, reliable, high-performing foundation for property and casualty data.

• Design the Databricks lakehouse architecture using a bronze/silver/gold approach on Delta Lake for policy, rating, claims, billing, reinsurance, and customer data.
• Plan and lead phased migration from Guidewire (Policy Center, Claim Center, Billing Center) and legacy systems into Azure Databricks, including historical loads
  , incremental/CDC loads,
  reconciliation
  , and decommissioning support
  .
• Model core property and casualty insurance entities:
  accounts
  , policies
  , insured risks and exposures
  , coverages
  , endorsements
  , premiums
  , claims
  , reserves
  , payments
  , and recoveries
  .
• Provide data for Actuarial and Finance use cases including written/earned/unearned premium
  , loss triangles
  , IBNR (incurred but not reported) inputs,
  loss ratios
  , and Schedule P and NAIC statutory reporting.
• Integrate third-party sources such as geospatial and weather data.
• Define and operationalize Unity Catalog design:
  metastore
  , catalogs
  , schemas
  , development/test/production environments
  , naming conventions,
  permissions
  , row/column-level security
  , and lineage
  .
• Manage pipeline orchestration and operations in Databricks, including Databricks Workflows (Lakeflow Jobs), scheduling, dependencies,
  retries
  , alerting and SLA monitoring
  , and production failure handling with runbooks
  .
• Own performance and cost work:
  Spark tuning
  , partitioning
  , liquid clustering
  , Photon
  , and cluster/SQL warehouse sizing
  .
• Partner with IT and a Platform Engineer on infrastructure for storage
  , networking
  , security
  , identity
  , IaC
  , and CI/CD
  .
• Set standards for pipelines,
  testing
  , code review
  , and documentation
  , and lead mentoring and review for Data Engineers and junior contractors.
• Collaborate with the data governance lead to implement access controls,
  PII masking
  , lineage
  , and data quality rules
  .
• Perform other duties as assigned.

• Associate degree or an equivalent combination of education and experience.
• 10+ years of data engineering experience, including 6+ years hands-on with Databricks (PySpark, Spark SQL, Delta Lake), plus experience leading or mentoring engineering teams.
• Databricks Data Engineer Professional certification;
  Azure data or solutions architect certification is preferred.
• Strong knowledge of Unity Catalog
  : governance model, permissions, row/column-level security, lineage, and system tables.
• Experience leading large data migrations from core insurance and legacy systems into a lakehouse or data warehouse, with property and casualty domain knowledge and familiarity with the policy and claims lifecycle and core P&C KPIs.
• Knowledge of data modeling including dimensional modeling, data vault, SCD patterns, and insurance data models.
• Expert technical skills across Spark internals and performance tuning,
  streaming (structured streaming, Auto Loader),
  CDC patterns,
  MERGE
  , and Lakeflow/Delta Live Tables
  .
• Expert-level SQL and Python
  , with software engineering practices such as testing, code review, and modular design.
• Knowledge of data infrastructure:
  ADLS Gen2
  , Azure Data Factory
  , Event Hubs
  , Key Vault
  , Entra ,
  VNets
  , and Private Link
  .
• Experience scheduling and monitoring…