Senior Data Engineer

Why This Role Matters

We are seeking a skilled and experienced Senior Data Engineer for our Data and AI team to contribute to the development of innovative AI-powered projects at The Washington Post. As a key member of the team, you will design, build, and deploy data solutions that power cutting‑edge AI technologies, including Generative AI, personalization, and recommendation systems, and revenue science applications. Your work will also support advertising insights generation and campaign infrastructure.

The role involves gathering requirements, writing code, testing, deploying, and maintaining software using best practices in development. You will collaborate on large‑scale projects, contribute to new features, products, and tools, and mentor junior developers.

• You are passionate about solving complex technical challenges and driving impactful projects that shape the user experience for a news media platform.
• You thrive as a natural problem‑solver, strong communicator, and proactive, curious individual who always seeks to learn and grow.
• You are motivated by the opportunity to tackle complex technical challenges, lead and influence a team, and collaborate with cross‑functional teams to drive user‑focused solutions.
• You have the freedom to explore emerging technologies and trends, and to identify new opportunities for growth and innovation.
• You are passionate about delivering high‑quality solutions that meet the needs of our users.

• Technical Leadership
  :
  Lead data engineering efforts using Python, Spark, Flink, and other data processing technologies, delivering scalable, secure, and reliable data pipelines, and contribute to the architecture of large, complex projects.
• Project Planning and Execution
  :
  Collaborate on gathering and analyzing project requirements to define technical solutions, and work with cross‑functional teams to integrate front‑end and back‑end solutions, ensuring seamless user experiences.
• Software Development
  :
  Write clear, maintainable code following best practices in version control, code reviews, testing, and ensure all software solutions are scalable, secure, and meet user needs.
• Collaboration and Communication
  :
  Work with data scientists, product managers, and designers to prioritize project requirements and communicate technical solutions to stakeholders.
• Mentorship and Coaching
  :
  Mentor and coach junior engineers, offer technical guidance and review code to ensure quality, and contribute to the growth of the engineering team.
• Project Management
  :
  Collaborate with Product, Design, Newsroom, and business stakeholders to manage and optimize project timelines, proactively identifying blockers and risks for timely delivery.
• Technical Expertise
  :
  Stay current with industry trends, suggest improvements to enhance performance, security, and scalability, and apply data engineering knowledge to drive innovation.
• Documentation and Knowledge Sharing
  :
  Contribute to technical documentation and design specifications and share knowledge and expertise to promote a culture of innovation and collaboration.

• Education and Experience
  :
  Bachelor’s degree in Computer Science, Software Engineering, or a related field;
  Master’s preferred. Equivalent professional experience considered. 3+ years of overall experience in ETL/ELT, data modeling, and querying large datasets using SQL.
• Technical Skills
  :
  Extensive experience with ETL development using Python and PySpark; AWS Data Lakehouse technologies (Redshift, Athena, EMR, Glue catalog);
  Pub/Sub and event‑driven architecture; DAG design and deployment with Apache Airflow; experience with Apache Iceberg, Postgres, MySQL, DynamoDB, Tableau, and Superset preferred. Experience with Google Ad Manager data and Ad Tech vendors is a plus.
• Cloud Infrastructure
  :
  Strong proficiency with AWS services (EC2, S3, RDS, Lambda, Batch);
  Docker and ECS highly preferred.
• Software Development
  :
  Experience with coding standards, code reviews, testing, and CI/CD pipelines.
• Performance Monitoring
  :
  Experience with observability tools (Splunk, Datadog) for maintaining application health and performance.
• Leadersh…