Senior Data Engineer

Senior Data Engineer

Department: Technology

Employment Type: Permanent

Location: Any UK Office Hub (Bristol / London / Manchester / Swansea)

Compensation: GBP 60,000 - GBP 80,000 / year

Our Senior Data Engineers enable public sector organisations to embrace a data-driven approach by providing data platforms and services that are high-quality, cost-efficient, and tailored to clients' needs. They develop, operate, and maintain these services. They make sure they provide maximum value to data consumers, including analysts, scientists, and business stakeholders.

As a Senior Data Engineer, you may play one or more roles according to our clients' needs. The role is very hands-on and you'll support as a senior contributor for a project, focusing on both delivering engineering work as well as upskilling members of the client team. At other points, you might play more of a technical architect role and work with the larger Made Tech team to identify growth opportunities within the account.

You’ll need to have a drive to deliver outcomes for users. You’ll make sure that the wider context of a delivery is considered and maintain alignment between the operational and analytical aspects of the engineering solution.

We are looking for candidates with a range of skills and experience, please apply even if you don’t meet all the criteria.

• Enthusiasm for learning and self-development
• Proficiency in Git (inc. Github Actions) and able to explain the benefits of different branch strategies
• Gathering and meeting the requirements of both clients and users on a data project
• Strong experience in IaC and able to guide how one could deploy infrastructure into different environments
• Owning the cloud infrastructure underpinning data systems through a Dev Ops approach
• Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop
• Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes) and the different use cases for them
• Ability to create data pipelines on a cloud environment and integrate error handling within these pipelines. With an understanding how to create reusable libraries to encourage uniformity of approach across multiple data pipelines.
• Able to document and present an end-to-end diagram to explain a data processing system on a cloud environment, with some knowledge of how you would present diagrams (C4, UML etc.)
• To provide guidance how one would implement a robust Dev Ops approach in a data project. Also would be able to talk about tools needed for Data Ops in areas such as orchestration, data integration and data analytics.
• Experience in improving resilience into a project by checking for software vulnerabilities and implement appropriate testing strategies (unit, integration, data quality etc.)
• Knowledge of SOLID, DRY and TDD principles and how to practically implement these into a project.
• Agile practices such as Scrum, XP, and/or Kanban
• Designing and implementing efficient data transformation processes at scale, both in batch and streaming use cases
• Owning the cloud infrastructure underpinning data systems through a Dev Ops approach
• Agile practices such as Scrum, XP, and/or Kanban
• People skills such as mentoring, supportive team player and performing line management duties
• To be able to demonstrate a commercial mindset when on projects to grow accounts organically with senior stakeholders

Experience in the following things isn’t essential, but it’s highly desirable!

• Working at a technology consultancy
• Working with Docker and virtual environments as part of the development and CI/CD process.
• Working with senior stakeholders to gather requirements and keep them engaged with
• Experience in working with a team of engineers using a variety of techniques such as pair programming or mob programming.
• Working with data scientists to product ionise advanced data deliverables, such as machine learning models
• Working knowledge of statistics
• Working with multidisciplinary digital and technology teams
• Working within the public sector