Mid-Senior Data Engineer

We are looking for a mid-experience Data Engineer to help modernize our data ingestion landscape. You will play a key role in migrating legacy ETL/ingestion processes to state-of-the-art cloud-native workloads on Microsoft Azure and Snowflake, enabling reliable, scalable, and observable data pipelines. You’ll work closely with data architects, analysts, platform teams, and business stakeholders to ensure high-quality data delivery across operational and analytical use cases.

• Design, build, and maintain robust ingestion pipelines (batch and near-real-time) from diverse sources such as databases, APIs, files, and event streams.
• Migrate legacy ETL/ELT processes to modern Azure and Snowflake patterns (re-platform and re-factor).
• Implement incremental loads, CDC patterns, schema evolution handling, and backfills/reprocessing strategies.
• Standardize and automate ingestion workflows using reusable frameworks, templates, and best practices.
• Develop cloud-native ingestion solutions leveraging Azure services such as Azure Data Factory/Synapse Pipelines, Azure Databricks and/or Spark, Azure Storage/ADLS Gen2, and optional event-driven services like Event Hubs.
• Build ingestion and loading patterns into Snowflake using Snowflake stages, file formats, COPY INTO, Streams/Tasks, and data modelling foundations for raw to curated data layers with dbt.
• Build components to capture streaming data sources.
• Develop real-time transformation pipelines and ensure timely delivery to consumer services.
• Develop shared service components, Python libraries, and integration templates to accelerate delivery across Data Engineering and Application teams.
• Follow guidance on integration best practices and ensure consistency across digital services.
• Implement data validation and quality checks for completeness, freshness, duplicates, and schema drift.
• Ensure pipelines are reliable and recoverable, incorporating idempotency, retries, re-runs, and alerting.
• Apply security best practices such as least privilege, secrets management, encryption, and secure connectivity.
• Observe data governance by following naming standards, applying data retention, classification, and documentation.
• Collaborate in agile delivery through code reviews, CI/CD, iterative release planning, and cross-team coordination.

• 4-6 years of hands-on data engineering experience, with a strong focus on data ingestion.
• Experience building production pipelines using Azure Data Factory, Databricks, and Synapse.
• Solid SQL skills and experience working with modern cloud data warehouses, ideally Snowflake.
• Proficiency in Python for data processing, automation, and pipeline utilities.
• Good understanding of data lake/lakehouse concepts and ingestion patterns.
• Infrastructure-as-Code exposure (Terraform) and CI/CD (Azure Dev Ops).
• Familiarity with orchestration frameworks such as Dagster.
• Able to prototype quickly while adhering to Group standards and controls.
• Clear communication with business stakeholders and technical teams.
• Experience with energy commodity trading is a real advantage.
• Ability to manage and prioritize multiple tasks in a fast-paced, deadline-driven environment.
• Strong problem-solving and troubleshooting skills.
• Self‑motivated, creative, highly organized, and proactive.
• Fluent in English (spoken and written).

By submitting your resume, you agree to the retention and use of your personal data by TSG for recruitment purposes, including sharing with our clients in the context of your application.