Grafana Observability SME | Observability Engineer; Grafana Solutions

Job Title:

Grafana Observability SME | Observability Engineer (Grafana Solutions)

Location:

Poughkeepsie, NY

Work Arrangement:
Onsite

Employment Type:

Contract

Duration: 6 months

Pay Range: $40.46/hr. to $43.08/hr. on W2 | $50.00/hr. to $56.00/hr. on C2C

Domain:
Information Technology | Enterprise Observability & Monitoring

• Full Grafana Cloud Stack Production Expertise:
  Hands-on architecture and operational experience with Grafana 12, Mimir, Loki, Tempo, Alloy, Beyla, Grafana Application Observability, and Unified Alerting.
• Advanced Querying & SLO Design:
  Expert-level proficiency authoring PromQL, LogQL, and Trace
  
  QL from scratch, including writing complex recording rules and Service Level Objective (SLO) queries.
• Open Telemetry (OTel) Practitioner:
  Mastery of OTLP, Open Telemetry collectors, and SDK/agent instrumentation across multi-language workloads (Java, .NET, Go, Python, and Node.js).
• eBPF Auto-Instrumentation:
  Deep production experience utilizing Grafana Beyla (or equivalent eBPF engines like Pixie, Cilium Tetragon) for zero-code telemetry gathering.
• Native Service Now ITOM Integration:
  Proven track record of integrating Grafana alerts into Service Now Event Management using native inbound integration patterns (moving beyond simple webhook-only models) with a solid grasp of AIOps event correlation and CMDB Configuration Item (CI) attachment.
• Git Ops & Dashboard-as-Code:
  Strong execution of dashboard-as-code patterns using Grafana provisioning tools, the Terraform provider, or Grizzly.
• Infrastructure Fluency:
  Experience deploying host telemetry agents at scale across multi-cloud and bare-metal platforms (On-Premises, AWS, Azure on both Linux and Windows).

• Experience implementing observability Fin Ops, such as cardinality control, log volume reduction, and storage retention tuning in Mimir/Loki.
• Familiarity with co-existing tooling landscapes, specifically Solar Winds (infrastructure/network) and Uptrends (synthetics), to engineer a unified "Single Pane of Glass" (SPoG).
• Background working within highly regulated sectors (e.g., Utilities, Energy, or Critical Infrastructure).
• Understanding of Service Now Common Service Data Model (CSDM) and Service Mapping frameworks.

We are seeking an elite Grafana Observability Subject Matter Expert (SME) with 12+ years of experience to own the end-to-end technical design, deployment, and institutionalization of a Grafana Cloud platform. Operating onsite in Poughkeepsie, NY, you will serve as the premier technical authority across a 50-application enterprise estate spanning diverse technology stacks (Java, .NET, Go, Python, Node.js) across hybrid on-prem and multi-cloud (AWS/Azure) environments.

This specialized role centers strictly on advanced application-level observability and APM—architecting high-performance telemetry pipelines, establishing Git Ops dashboard standards, and engineering native integrations into Service Now ITOM for intelligent event correlation and automated incident management.

• Platform Architecture:
  Architect and configure the complete enterprise Grafana Cloud stack (Grafana 12, Mimir, Loki, Tempo, Alloy, Beyla), defining organizations, folders, teams, and robust role-based access controls (RBAC).
• Instrumentation Strategy:
  Define and deploy the application instrumentation roadmap, balancing zero-code pathing via Beyla eBPF for lower-complexity applications and deep custom OTel SDK/agent integrations for high-complexity core workloads.
• Log Pipeline & Query Engineering:
  Engineer structured log pipelines through Alloy (handling JSON, Log4j, Serilog, NLog, Winston, etc.), tuning parsing rules and writing optimized PromQL, LogQL, and Trace
  
  QL rules for low-noise, high-actionability alert feeds.
• Service Now ITOM Integration:
  Co‑own the blueprint and deployment of the native Grafana‑to‑Service Now Event Management inbound flow, implementing "deny by default" allow-list governance, automated deduplication, AIOps correlation, and severity‑mapped CMDB CI attachments.
• Single Pane of Glass (SPoG) Creation:
  Design and deliver a tiered enterprise visualization plane that elegantly displays application-level telemetry while embedding contextual routing boundaries out to peripheral tools like Solar Winds and Uptrends.
• Quality Assurance & Knowledge Transfer:
  Act as the ultimate signing authority on all system deliverables, from solution architecture documentation and instrumentation runbooks to testing scripts. Conduct structured handovers to ensure successful post-deployment support by the client’s internal operations team.

Medical | Dental | Vision | 401(k)

We are an equal opportunity employer, and all qualified applicants will receive consideration for employment.