Senior Data Engineer | Energy Trading | Agnostic - Cloud-native AI Centric Data Platform

Senior Data Engineer | Energy Trading | Agnostic - Cloud-native AI Centric Data Platform | Next-Gen Lakehouse | Up to £100K + Bonus + Benefits

Senior Data Engineer is required for a leading global Energy & Commodities trading house — one of the world’s largest and most diverse general trading companies. Their trading ecosystem spans multiple commodities and markets, creating high-volume, high-velocity data challenges ideally suited to Click House and modern data engineering.

As a Senior Data Engineer, you will play a pivotal role in choosing the right tooling, building and scaling a cloud-native AI-driven analytical data platform. You’ll be a senior member of the Core Engineering team, shaping the architecture and implementing high-performance data solutions used directly by trading, risk, analytics and operational teams.

Whilst the firm has agreed to do an Azure Databricks POC platform in Q1 2026, with so much of their data on‑prem, they are still showcasing different Cloud-native platforms, and therefore are looking for someone who is excited to build Proof of Concepts in the right platform for the business requirements.

This is a hands‑on position where you’ll be working with a high-skilled small team of engineers while designing and developing a next‑generation lakehouse architecture.

Azure Databricks & Click House will form the backbone of examples of two systems, supporting ultra-fast analytics, market-data ingestion, tick‑data storage, and real‑time insights for traders and quantitative teams.

• Salary: Up to £100,000 per annum + Bonus & Benefits

• Design, build, and optimise high-performance Click House / Azure Databricks / AI Cloud-native Data clusters for large‑scale analytical workloads across commodities and energy trading.
• Develop ingestion frameworks for market data, pricing data, tick data, and trade/position datasets, leveraging Click House’s columnar storage and real‑time querying.
• Architect and maintain a modern lakehouse platform integrating Click House with Databricks, Snowflake, and Azure-based services.
• Define and enforce best practices for data modelling, partitioning, compression, and query optimisation tailored for Click House.
• Build scalable Python-based ETL/ELT pipelines and data services that deliver robust, low-latency data to trading, risk, and analytics teams.

If you are interested in this role, please send your CV to ross.

Location:

London, England, United Kingdom

Referrals increase your chances of interviewing at Virtue Tech Recruitment Group by 2x.