Principal Software Development Engineer - Observability

Expedia Group brands power global travel for everyone, everywhere. We design cutting‑edge tech to make travel smoother and more memorable, and we create groundbreaking solutions for our partners. Our diverse, vibrant, and welcoming community is essential in driving our success.

To shape the future of travel, people must come first. Guided by our Values and Leadership Agreements, we foster an open culture where everyone belongs, differences are celebrated and we know that when one of us wins, we all win.

Our Technology Team partners with teams across Expedia Group to create innovative products, services, and tools to deliver high-quality experiences for travelers, partners, and our employees. A singular technology platform powered by data and machine learning provides secure, differentiated, and personalized experiences that drive loyalty and traveler satisfaction.

As a Principal Engineer, you will be part of an agile development team with deep expertise in cloud, distributed systems, and observability. You will play a pivotal role in crafting the strategic technical goals for our group. The main effort will involve leading the architecture, design, and implementation of a centralized, scalable, and cost‑effective observability platform used by all engineering teams across Expedia.

You will provide technical leadership for a dynamic engineering organization and work alongside talented product managers and other technical leaders to deliver best‑in‑class capabilities to our developer community.

• Architect and Build Core Telemetry Pipelines:
  Lead the design and implementation of highly scalable and resilient telemetry pipelines for logs, metrics, and traces. Evolve our platform to handle a 10x increase in data volume while maintaining performance and cost‑effectiveness
• Drive Open Telemetry Adoption:
  Spearhead the strategy, rollout, and support for the Open Telemetry collector across thousands of services. Develop best practices and automated configurations to ensure seamless and consistent data collection
• Implement Platform Governance and Optimization:
  Design and build capabilities for data governance, cost allocation, and resource management within the observability platform. Define and implement SLOs for the platform itself and create tools to help teams manage their observability costs
• Elevate the Practice of Observability:
  Act as a thought leader, driving the adoption of observability best practices across the engineering organization. Improve the developer experience by unifying tooling (e.g., Grafana, Datadog, Splunk), documentation, and service lifecycle management within our internal developer portal
• Automate Infrastructure Lifecycle:
  Author and maintain production‑grade Infrastructure as Code (IaC) using tools like Terraform and/or Crossplane. Eliminate manual toil by automating cluster provisioning, dependency upgrades, and incident remediation workflows
• Technical Leadership and Mentorship:
  Act as a force multiplier. Mentor senior engineers on the team, lead architecture review sessions, and author RFCs to build consensus on significant technical decisions. Your influence will extend beyond the team to application developers and SREs
• Production Debugging:
  Serve as the final escalation point for complex, cross‑cutting production incidents related to the observability platform, from telemetry agent bugs to data correlation failures in our distributed systems
• Collaborate and Innovate:
  Explore and utilize a wide variety of technologies and tools, such as (but not limited to) Go, Java, Python, AWS, Kubernetes, Open Telemetry, Prometheus, Grafana, Datadog, Splunk, Clickhouse

• Bachelor’s or Master’s degree in Computer Science or a related technical field, or equivalent practical experience
• 10+ years of experience in software engineering, with a focus on building and operating large‑scale distributed systems, infrastructure automation, or configuration management
• Deep expertise in observability principles and the "three pillars": logs, metrics, and traces
• Strong hands‑on…