Data Engineer

Senior Data Engineer (Contract)

Duration: 6 Months (Possibility for extension)

Location: London/Hybrid (3 days per week on site)

Rate: A highly competitive Umbrella Day Rate is available for suitable candidates

We are seeking a highly skilled Senior Data Engineer with strong expertise in Python and Databricks to design, build, and optimise scalable data pipelines and platforms. You will play a key role in enabling data-driven decision‑making by delivering reliable, efficient, and high‑quality data solutions.

• Design, develop, and maintain scalable data pipelines using Python and Databricks
• Build and optimise ETL/ELT workflows leveraging Databricks (Spark, Delta Lake)
• Develop robust data models and data architectures on Lakehouse platforms
• Implement and manage data workflows within the Databricks ecosystem
• Ensure high data quality, governance, and reliability across pipelines
• Optimise performance of large‑scale distributed data processing
• Collaborate with data scientists and analysts to support analytics and ML
• Implement monitoring, logging, and alerting for data systems
• Mentor junior engineers and drive engineering best practices

• Extensive experience in data engineering
• Strong programming expertise in Python
• Hands‑on experience with Databricks (workflows, notebooks, jobs, Delta Lake)
• Strong experience with Apache Spark / Py Spark
• Experience building scalable ETL pipelines
• Proficiency in SQL and relational databases
• Experience with cloud platforms (AWS / Azure / GCP)
• Experience with modern data warehousing solutions (e.g., Snowflake, Big Query, Redshift)
• Familiarity with orchestration tools (e.g., Airflow, Databricks Workflows)
• Strong understanding of data modelling and lakehouse architecture
• Databricks certifications (e.g., Databricks Certified Data Engineer)
• Experience with streaming data pipelines (Kafka, Structured Streaming)
• Experience with CI/CD and Dev Ops practices
• Knowledge of Delta Lake features (time travel, CDC, optimisation)
• Experience with containerisation (Docker, Kubernetes)
• Exposure to ML pipelines on Databricks

Candidates will need to show evidence of the above in their CV in order to be considered.

We respect and appreciate people of all ethnicities, generations, religious beliefs, sexual orientations, gender identities, and more. We do this by showcasing their talents, skills, and unique experience in an inclusive environment that helps them thrive.