Senior Data Engineer; ED

Referment is working with a fast-growing fintech that is building a zero-fee payments platform for merchants, funded by well-known venture investors and already processing at a significant run rate with millions of customers.

You'll join a data team that runs an event-driven platform on AWS and Snowflake. Product services emit events through Event Bridge and Kinesis Firehose into S3, auto‑ingested into Snowflake via Snowpipe, while batch and third‑party data lands through Snowpark‑deployed Python procedures. All infrastructure is defined in Terraform/Terragrunt, pipelines are built in Python, CI/CD runs on Git Hub Actions, and Datadog provides end‑to‑end pipeline observability.

• Own and evolve the event ingestion path — Event Bridge to Firehose to S3 to Snowpipe — including partitioning and throughput tuning as event volumes grow across streams such as fraud, payments and merchant.
• Design and deploy Snowpark‑based Python procedures for file‑based and third‑party data loads, scheduled and monitored as Snowflake Tasks.
• Work with product engineers to manage upstream schema changes and late‑arriving data without breaking downstream consumers.
• Own and extend Terraform/Terragrunt modules for ingestion infrastructure across multiple environments.
• Design Datadog monitors and alerting for Firehose, Event Bridge, DLQs and Snowpipe, and triage ingestion failures when they occur.
• Manage PII handling, masking, data governance and access in Snowflake, supporting audit‑ready reconciled data for Finance and Compliance.
• Build privacy‑first data pipelines using dbt and Snowflake Dynamic Tables, with testing and monitoring that catches issues before they reach the business or the Analytics Engineering team.

• Hands‑on experience with streaming and event ingestion on AWS — Kinesis, Event Bridge or equivalents — plus S3 partitioning and queue/DLQ patterns.
• Terraform experience is essential; owning modules across environments rather than just consuming them. Terragrunt is a plus.
• Strong SQL on Snowflake, including Snowpipe, Tasks and Snowpark.
• Proven dbt experience building well‑tested SQL and Python models.
• Experience with Git Hub Actions (or an equivalent) and with secrets management, PII and sensitive data handling in Snowflake.
• Python skills for pipeline and infrastructure code — typed models, testing and CI‑deployed services rather than just scripting.

You’ll need a pragmatic, systems‑level mindset and the ability to explain pipeline failures, infrastructure trade‑offs and data‑flow issues clearly to both engineers and non‑technical stakeholders. This role suits someone who takes full ownership of what they build, from design through to production reliability.

This could suit a data engineer at a scale‑up or a data‑heavy fintech who has outgrown ticket‑driven work and wants to own an entire ingestion platform end to end.

#Referment