Sr. Data Engineer, AI-Augmented

What You Can Expect In This Role

Enterprise data engineers in this role support the real-time data movement and shared data services layer of the Enterprise Data & Integration platform. This team is responsible for the day-to-day operational health of Change Data Capture feeds, business event streams, Kafka topics, Snowpipe ingestion, and shared data services that enterprise data products depend on.

This is a hands‑on operational engineering role focused on production support, incident response, root‑cause analysis, reliability, and continuous improvement. The ideal candidate is calm under pressure, methodical in troubleshooting, comfortable working across logs, SQL, Kafka, Snowflake, and Azure, and able to drive complex data pipeline incidents to resolution.

• Monitor the health of CDC pipelines and shared data services in production, including consumer lag, throughput, error rates, dead‑letter queues, Snowpipe backlogs, and end‑to‑end latency.
• Triage and resolve data pipeline incidents by identifying whether the issue is related to the source system, data movement layer, Kafka, Snowpipe, Snowflake, or downstream consumers, and engage the appropriate owners.
• Serve in the on‑call rotation for SEV‑1 and SEV‑2 data incidents in alignment with the Data Pipeline Incident Response runbook.
• Diagnose schema‑evolution issues in Confluent Schema Registry, including incompatible changes, deserialization failures, and consumer poison messages, and coordinate fixes with publishing teams.
• Replay, reprocess, and backfill data through Kafka and Snowpipe when upstream systems recover from outages, resend data, or provide corrections.
• Investigate data‑quality issues reported by consumers, including missing records, duplicates, late‑arriving data, out‑of‑order events, and mismatched counts between source systems and hist_* / curr_
  * zones in Snowflake.
• Tune and right‑size pipeline components for performance, reliability, and cost efficiency, including Kafka partitions and consumers, Snowpipe warehouses, and Azure Function scale settings.
• Maintain and improve operational tooling, including monitoring dashboards, alerting, and replay utilities across Confluent, Snowflake, Azure Monitor, and Application Insights.
• Write, maintain, and improve operational runbooks so recurring issues can be resolved faster and more effectively by first‑level support teams.
• Apply small code and configuration changes to Python services, Azure Functions, SQL, and dbt models in support of operational fixes, while partnering with build‑side data engineers on larger enhancements.
• Manage routine platform operations, including schema registry hygiene, topic lifecycle, credential rotation, certificate renewals, and access requests via Ivanti.
• Communicate clearly with application teams, data product owners, and business stakeholders during and after incidents, including status updates, root cause summaries, and follow‑up actions.
• Contribute to postmortems and feed lessons learned back into engineering standards, support practices, and the EDI Architecture Wiki.

• College Degree in the field of computer science, information science, management information systems Preferred
• Minimum 8 years' IT development experience or equivalent Preferred
• Effective verbal and written communications skills and the ability to communicate with business partners and other IT staff
• Problem solving skills sufficient to perform research and recommend a proposed solution to problems
• Able to work on multiple tasks and meet established deadlines
• Able to effectively direct and coordinate the work of other team members on a project without having HR management responsibility for them
• Knowledge of computer programming languages as required for the system

• Criminal background check required.

USD $84,721.00 - USD $ /Yr. Range includes data points from multiple labor markets. Specific range is dependent on the labor market where the incumbent will be hired to perform the position. Starting salary is dependent on candidate qualifications and experience. For a narrower salary range specific to your labor market,…