Principal Data Engineer

Disney Entertainment and ESPN Product & Technology is hiring a Principal Data Engineer to lead Identity and Device data engineering for marketing measurement and analytics.

• Define and drive technical strategy for Identity & Device data products in Marketing and Consumer Data, including pipeline architecture, platform patterns, and a modernization roadmap in partnership with onshore squad leaders and enterprise architecture.
• Own end-to-end delivery for complex, high-impact initiatives spanning Identity
  360
  , device/consumer identifier datasets, and dependent Marketing data products, including technical approach, sequencing, and engineering quality standards for other engineers.
• Act as principal subject matter expert and escalation point for identity foundational data, covering data modeling
  , batch and near-real-time processing
  , performance at scale
  , and production reliability
  .
• Participate in an on-call rotation and lead incident response for critical pipelines.
• Set and enforce engineering standards for the Identity & Device team and Marketing Data Engineering, including code quality
  , CI/CD
  , observability
  , and data governance aligned to GDPR
  , RTBF
  , and CCPA
  , along with reusable platform patterns across Databricks
  , AWS
  , and orchestration tooling.
• Mentor and develop engineers through design reviews
  , pairing
  , and coaching
  , building depth in big-data engineering, identity domain knowledge, and operational excellence.
• Collaborate cross-functionally with Product
  , QA
  , Operations
  , Privacy/Legal
  , and peer Marketing data squads (Data Collection, Activation, Acquisition) to align requirements, resolve cross-domain dependencies, and represent technical decisions to stakeholders.

• 10+ years of related big data engineering experience, including modeling and building large data pipelines; strong experience with Python, Scala, or Java
  ; strong AWS fundamentals (S3, Lambda, IAM) preferred.
• 5+ years hands-on experience with performance tuning at scale using distributed systems such as Hadoop or Databricks
  , with query/process tools such as Spark
  , Presto
  , and Hive at the petabyte level.
• Experience with at least one major MPP or cloud database technology:
  Snowflake
  , Redshift
  , Big Query
  , or Databricks
  .
• Solid understanding of data modeling and data warehousing standard methodologies and practices.
• Operational experience in a multi-system/services production environment as a senior data engineer and point-of-contact/SME.
• Strong problem-solving with attention to detail, plus excellent analytical and communication skills.
• Experience with data integration and orchestration toolsets (e.g.,
  Airflow
  ) and CI/CD
  .
• Demonstrated ability with cloud infrastructure technologies, including Terraform
  .
• Strong understanding of slowly changing dimensions (SCD tables) and building scalable, performant datasets.
• Bachelor’s degree in computer science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience.

• Python, Scala, Java
• AWS, S3, Lambda, IAM
• Hadoop, Databricks, Spark, Presto, Hive
• Snowflake, Redshift, Big Query
• Airflow, CI/CD, Terraform
• GDPR, RTBF, CCPA

• Bonus and/or long-term incentive units may be provided as part of the compensation package.
• Full range of medical, financial, and/or other benefits.

Location: New York, NY (onsite)

Compensation: USD 187, per year

Experience: 10+ years

PAY TRANSPARENCY: The hiring range for this position in SFO, CA is $187,900 - $252,000 per year and New York is $179,700 - $241,000 per year.

Note: This position is with Disney Entertainment & Sports LLC, which is part of a business we call Disney Entertainment and ESPN Product & Technology.