Principal Data Engineer - Databricks

Company: Alaska Airlines

The Principal Data Engineer is the sole subject matter expert in designing, optimizing, implementing, and advancing Alaska Air Group’s Enterprise Data platform using Databricks. As an individual contributor, this role defines long‑term strategy for driving innovation, mentoring teams, partnering on the best technical implementation of Databricks, and ensuring the platform is scalable, secure, and aligned with Business and ITS goals.

• Act as lead subject matter expert in designing and implementing high‑performance batch or real‑time data pipelines using Apache Spark, Delta Live Tables, and Structured Streaming on Databricks.
• Collaborate with data scientists, analysts, and engineering teams to deliver robust, production‑grade solutions.
• Define and enforce best practices for workspace governance, job orchestration, and CI/CD integration.
• Optimize cluster configurations, spark parameters, and job execution plans for cost efficiency and speed.
• Evaluate and implement new Databricks features to enhance platform capabilities.
• Exercise considerable latitude and initiative to troubleshoot and resolve performance bottlenecks, memory issues, and streaming latency challenges.
• Lead development and mentor engineers on real‑time streaming solutions, spark internals, delta table optimization strategies, and advanced debugging techniques.
• Implement and manage Unity Catalog for centralized governance, data lineage, and fine‑grained access control.
• Monitor Databricks costs, identify, evaluate, and drive implementation of cost‑reduction opportunities.
• Lead efforts to standardize tagging and metadata practices across the AAG Databricks environment.
• Focus on continuous improvement in system reliability through automation and monitoring, including implementation of automated unit testing.
• Collaborate directly with Enterprise Architecture and Security to develop and plan platform architecture and security strategies.
• Act as a forward thinker to help ensure that new capabilities are evaluated, implemented, and operationalized.

• 7 years of experience in data engineering, big data platforms, and analytics, including independent work on large‑scale data architectures.
• Bachelor’s degree, or two years of relevant training/experience in lieu of this degree.
• Expert‑level proficiency in Databricks and Apache Spark.
• Deep understanding of Databricks configurations and hands‑on experience with real‑time streaming using Structured Streaming and Delta Live tables.
• Strong coding skills with Python, SQL, and PySpark.
• Expert knowledge and experience in Infrastructure as Code (ARM, Terraform) and CI/CD pipeline (Azure Dev Ops and/or Git Hub preferred).
• Proven ability to optimize data pipelines for performance, reliability, and cost.
• Familiarity with Unity Catalog and Lakehouse Federation.
• Ability to work collaboratively with cross‑functional teams to understand objectives and write technical specifications in a lead role.
• Experience with Kafka.
• Excellent communication skills and ability to collaborate or contribute to technical debate and design choices with a variety of team members.
• High school diploma or equivalent.
• Minimum age of 18.
• Must be authorized to work in the U.S.

• Familiarity with Azure and MLflow.
• Certifications such as Databricks Certified Data Engineer Professional or Azure Solutions Architect Expert.
• Experience in Agile (Scrum) or Kanban development methodologies.
• Experience estimating effort and establishing timelines for projects.
• Experience conveying knowledge to diverse groups of people (classes, brown‑bag meetings, seminars).

Embody our values to own safety, do the right thing, be caring and kind, and deliver performance.

$138,500 - $207,750 / year

Pay will be based on multiple factors, including and not limited to location, relevant experience/level and skillset while balancing internal equity relative to other Alaska/Hawaiian/Horizon employees. Alaska/Hawaiian/Horizon is committed to fair, unbiased compensation along with competitive benefits in all locations in which we operate.

• Free…