Senior Data Engineer - Python, Databricks & Counterparty Credit Risk

Location: London
- Hybrid

We are recruiting for an experienced Senior Data Engineer to join a leading global banking organisation, working across a highly technical data environment supporting Counter party Credit Risk (CCR), Credit Risk and wider risk data
.

This is a hands-on engineering position requiring strong expertise across Python, PySpark, SQL and Databricks
, alongside experience designing, building and optimising scalable data pipelines and Lakehouse solutions.

Candidates should ideally have previously worked with Counter party Credit Risk or Credit Risk data
, including areas such as counter party/exposure data, risk analytics, regulatory risk or associated banking risk datasets.

• Design, develop and maintain scalable data pipelines using Python, PySpark and Databricks
• Build and optimise ETL/ELT workflows using Spark and Delta Lake
• Engineer and transform data supporting Counter party Credit Risk, Credit Risk and wider risk functions
• Work with datasets relating to areas such as counter parties, credit exposures, limits, collateral and risk analytics
• Design robust data models and architectures across Lakehouse / Medallion environments
• Develop and manage Databricks notebooks, jobs and workflows
• Optimise large-scale distributed data processing and troubleshoot Spark performance
• Implement strong data quality, reconciliation, governance and monitoring controls
• Build reliable logging and alerting across production data pipelines
• Integrate data platforms with Power BI dashboards and wider applications
• Support CI/CD and engineering best practices across development, testing and production environments
• Work closely with Risk, Technology, Analytics and other senior stakeholders

• 10+ years' experience across Data Engineering or closely related data disciplines
• Strong hands-on Python development experience
• Strong PySpark / Apache Spark engineering experience
• Extensive hands-on experience with Databricks
  , including notebooks, jobs/workflows and Delta Lake
• Advanced SQL and relational database experience
• Strong track record designing and building scalable ETL/ELT data pipelines
• Experience with Delta Lake, Lakehouse and/or Medallion architectures
• Experience handling banking risk data
  , ideally within Counter party Credit Risk (CCR), Credit Risk, counter party exposure or closely related risk domains
• Strong understanding of data modelling, data quality and production data engineering
• Cloud experience across Azure, AWS and/or GCP
• Experience with modern data platforms/warehouses such as Snowflake, Big Query or Redshift
• Experience integrating data platforms with Power BI
• Experience with orchestration such as Databricks Workflows or Airflow

• Direct experience engineering data for Counter party Credit Risk
  , including exposure/counter party datasets
• Knowledge of areas such as PFE, EPE, EAD, CVA/XVA, SA-CCR, collateral, netting or counter party limits
• Previous experience within an investment bank, global bank or capital-markets environment
• Databricks Data Engineer certification
• Kafka / Structured Streaming experience
• Delta Lake CDC, Time Travel and optimisation
• CI/CD, Git and Dev Ops experience
• Docker / Kubernetes experience
• Experience exposing data through APIs or supporting React-based front-end applications
• Experience supporting analytics or ML workloads within Databricks

We are particularly interested in genuinely hands-on Senior Data Engineers who can demonstrate the Databricks and Python solutions they have personally designed, built, optimised and supported in production.