Senior Data Engineer

Main duties of the job

• To develop, maintain and support the corporate data warehouse and Trusted Research Environment. This will include and is not limited to data engineering tasks, data pipeline development, and resolving performance issues.
• To work with our cloud hosting provider to ensure effective resource utilisation and consumption patterns.
• To develop complex data structures, data models and Extract Transformation and Load processes using Azure Synapse Analytics and related Azure services to support the trust Data warehouse at the direction of the Database Architect and Data Infrastructure Manager.
• To apply data engineering techniques including data wrangling, cleansing, transformation and integration to ensure high-quality data pipelines and datasets.
• Develop, implement, and assist in the maintenance of a data asset catalogue to facilitate data discovery.

For further information on this role, please see the attached detailed Job Description and Person Specification

• To work as part of the Information Development team to develop, support, and enhance the corporate data warehouse environment.
• Support the integrity of the data warehouse and contribute to the design and implementation of adequate back-up solutions.
• Contribute to business continuity and disaster recovery planning.
• Maintain an in-depth knowledge of Microsoft Azure Synapse Analytics, and related Azure data services, and provide expert advice regarding their application, act as a point of escalation for investigation and resolution of complex technical issues.
• Lead the development and implementation data pipelines using Synapse Pipelines and other ETL/ELT tools to ingest, transform and load data from various sources.
• Act as the SME for data engineering, providing authoritative input on schema design, data quality, and interoperability.

• Educated to degree level (or equivalent qualification/experience)
• Evidence of on-going professional development
• Microsoft certified:
  Azure Fundamentals, Data Fundamentals, Database Administrator, Designing & implementing Dev Ops Solutions

• Significant experience in a database developer role in a multi-professional organisation
• Evidence of having developed complex database systems
• Evidence of having administered Azure data warehouses including but not limited to: SQL Server including SSIS, T‑SQL, Data Factory/ Synapse Pipelines, Azure Dev Ops, including Repos & Test Plans, Azure Data Catalogue, Serverless application development
• Knowledge and understanding of modern data analytics platforms including but not limited to:
  Python, R, Power
  
  BI, Office 365
• Experience of working within an Agile and Continuous Integration environment
• Knowledge and experience of working with sensitive information within the NHS
• Knowledge and understanding of Machine Learning data pipelines

• A strong commitment to customer service delivery and excellent organisational skills
• Ability to evaluate and respond to rapidly changing operational situations, often under pressure and with changing and sometimes conflicting priorities
• Excellent communication skills especially when dealing with complex technical information and colleagues who have limited IT knowledge and understanding
• Ability to document technical procedures/standards
• An in depth understanding of schema creation for databases & management of permissions

• Self‑motivated: generates ideas and carries them forward, keen to deliver results and take ownership of work
• Self‑disciplined: able to work on own initiative, works to high standards
• Team working ability to work and contribute well within a team
• Good communicator: able to communicate clearly and effectively, both verbally and in writing

This post is subject to the Rehabilitation of Offenders Act (Exceptions Order) 1975 and as such it will be necessary for a submission for Disclosure to be made to the Disclosure and Barring Service (formerly known as CRB) to check for any previous criminal convictions.

£63,665 to £70,887 a year per annum inc HCAS