Senior Observability Engineer

Benefits Overview

A career at CoBank can offer you the opportunity to make a personal impact on the people and communities where we do business. In order to be the best, we hire the best!

Benefits Offered by Co Bank

• Careers with a purpose
• Time-Off Packages, 15 days of vacation, 10 paid sick days and 11 paid holidays
• Competitive Compensation & Incentive
• Hybrid work model: flexible arrangements for most positions
• Benefits Packages, including Medical, Dental and Vision coverage, Disability, AD&D, and Life Insurance
• Robust associate training and development with CoBank University
• Tuition reimbursement for higher education
• Outstanding 401k: up to 6% matching and additional 3% non-elective contribution & Student Loan Match
• Community Impact:
  United Way Angel Day, Volunteer Day and Associate Directed Contribution
• Associate Resource Groups: creating a culture of respect and inclusion
• Recognize a fellow associate through our GEM awards

Are you a seasoned professional with a passion for observability, telemetry, and system reliability? Do you excel in a dynamic, collaborative, and inclusive work environment? At CoBank, we are seeking a Senior Observability Engineer to help design, implement, and scale enterprise observability capabilities across both on-premises and cloud environments. In this role, you will play a key part in defining and advancing CoBank's observability strategy, with an initial focus on instrumentation and telemetry for on-prem systems, evolving into cloud-native observability within AWS.

You will partner closely with infrastructure and application teams to enable visibility across systems and services, ensuring reliable, performant, and measurable platforms. As a Senior Observability Engineer at CoBank, you will be responsible for implementing and optimizing observability solutions using platforms such as Splunk and Splunk Observability Cloud, while supporting modern telemetry standards including Open Telemetry. You will also contribute to building scalable ingestion pipelines, improving signal quality, and enabling actionable insights across infrastructure and applications.

• Designs, implements, and maintains enterprise observability solutions across on-prem and AWS environments.
• Develops and enhances monitoring, alerting, logging, and tracing capabilities using Splunk and related platforms.
• Implements instrumentation standards using Open Telemetry and other telemetry frameworks.
• Integrates telemetry from diverse systems (infrastructure, applications, and platforms) into centralized observability solutions.
• Collaborates with application teams to enable APM, distributed tracing, and performance monitoring.
• Builds and manages telemetry pipelines and collectors (e.g., Open Telemetry Collector or similar tools).
• Monitors system health and proactively identify reliability and performance issues.
• Optimizes telemetry ingest, data quality, and alert effectiveness to reduce noise and cost.
  
  Troubleshoots issues across infrastructure, applications, and observability platforms.
• Partners with cross-functional teams to establish and promote observability best practices.
• Stays current with emerging trends and technologies in observability, APM, and SRE practices.

• High school diploma or GED required
• Bachelor's Degree in computer science, information systems, or a related discipline preferred

• 5 years of experience in Dev Ops, SRE, cloud engineering, or IT operations required
• 3 years of experience implementing or supporting observability/monitoring solutions required
• Hands-on experience with Splunk (search, dashboards, alerting, ingest).
• Experience with Open Telemetry or similar instrumentation frameworks.
• Experience supporting observability across hybrid environments (on-prem and cloud; AWS preferred).
• Experience integrating telemetry from enterprise systems such as infrastructure platforms, virtualization, or cloud services.
• Familiarity with telemetry pipelines or aggregators (e.g., Open Telemetry Collector, Logstash, Fluentd) preferred.
  
  Experience supporting application teams with monitoring, APM, and instrumentation preferred.
• Exposure to open-source observability tools such as Prometheus, Grafana, or ELK stack preferred.
• Experience with Splunk Observability Cloud, Datadog, Dynatrace, or similar platforms preferred.
• Experience optimizing telemetry ingest, alerting strategies, or data retention is a plus preferred.
• Strong problem-solving skills and ability to work in a fast-paced environment preferred.
• Ability to apply independent judgment on most decisions and work with minimal guidance preferred.
• Demonstrates strategic thinking and ownership of technical outcomes preferred.
• Strong collaboration skills across engineering, operations, and development teams preferred.
• Enhances relationships and builds partnerships with cross-functional teams including infrastructure, application development, and security. Effectively communicates complex observability concepts to technical and non-technical…