Senior Data Engineer

• Interface with business stakeholders, understand their data and analytics needs, establish requirement

in alignment with data product owners and align on delivery plan.

• Lead discovery of the current reporting and data landscape: inventory sources, ETL processes, upstream/downstream dependencies, and existing metrics/definition
• Translate stakeholder requirements and current-state findings into a target pipeline design (batch and near-real time where applicable), including data flow diagrams, lineage, and logical/physical model
• Develop and maintain end to end data pipelines: author, tune, and optimize Python and SQL code for extraction, transformation, and loading (ETL/ELT), including incremental loads, partitioning, and CDC patterns
• Build reusable Python libraries and parameterized pipelines to accelerate delivery and ensure consistency
• Implement data quality, validation, and reconciliation processes: automated tests, reconciliation scripts, anomaly detection, and monitoring/alerting to ensure trust in the data
• Define and codify metric logic, mapping, and lineage; implement validation tests and documentation for each metric to support governance and auditability
• Own UAT for new/changed data assets: create test plans, run tests, document findings, and provide pass/fail recommndations to IT and engineering partners
• Prepare rollout materials and lead change adoption: runbooks, operational runbooks, deployment guides, knowledge transfer sessions, and training for analysts and business users
• Support table retirement and technical debt reduction by coordinating deprecation plans and ensuring no regression in reporting
• Collaborate in an Agile environment with peers and stakeholders; participate in planning, standups, and retrospectives to deliver iterative value
• Rigorously create and maintain document such that anyone can understand the design, logic, code, acceptance testing, data quality testing, and outcomes. Document prototypes in detail, including learnings of what doesn’t work.
• Effectively communicate analysis, reports, project status, milestones etc. in coordination with data product owners, program managers and scrum maters.
• Implement straight through, exception-based data product designs meeting data governance standards, including reporting. Perform necessary data quality checking tasks, prepare data quality reports, establish necessary data security measures to prepare model ready data.