Lead Data Engineer

About us At National Grid, we keep people connected and society moving. But it’s so much more than that. National Grid provides the environment, support, and purpose that enable us to make real impact. As we drive momentum in the energy transition for all, we’re committed to ensuring none of our customers are left in the dark.

This role operates under a hybrid working model and is based out of the Warwick, UK office.  Hybrid work expectations are subject to business needs and company policy.

About the Role To work with Electricity Transmission to deliver actionable insight that drives business performance. Collaborating with key stakeholders the Associate Data Developer will work to understand key business challenges, help determine the most impactful insight and translate this into business requirements. They will support the management of insight delivery across the full software lifecycle — including development, testing, CI/CD and deployment, consider the business readiness for insight implementation and undertake necessary engagement to ensure insights drive action and are continuously improved.

The role combines strong analytical capability with practical experience of building data products and dashboards on cloud platforms (particularly Azure).What You'll Do
• Understanding of data fabric, lake and warehousing principles and full project involvement in one or more major technology platforms, e.g. Snowflake, Azure etc.
• Development of the Extract Transform Load (ETL) for the relevant data insights platform
• Working in partnership with ET teams, across the relevant areas of ET data to undertake advanced data mining and manipulation to understand the quality and completeness of data to advise and define key changes to technology architecture & data models to facilitate performance measurement & business insight.
• Lead the development of new installations and patches for the relevant insight platform and support upstream system developers and downstream stakeholders
• Support the data flow and mapping for solution design of IT Projects
• Produce support documentation ranging from requirement capture to end delivery
• Transform the quality of ET’s recorded information so it is universally recognised as the single source of the truth which is consistently utilised by the business without the need for extraction from core systems to cleanse before use
• Design, implement and maintain CI/CD pipelines, automation and test frameworks for data pipelines, models and dashboard deployments (e.g. Azure Dev Ops, Git Hub Actions, or equivalent).
• Follow and promote good software lifecycle practices: version control, code review, automated testing, release management and rollback processes.
• Implement monitoring, logging and observability for data pipelines and dashboards (for example Application Insights, logging frameworks and pipeline metrics) and act on alerts.
• Work with ML/agentic automation teams to integrate or evaluate agent-based workflows where applicable (e.g. controlled LLM agents, orchestration of multi-step automated tasks).
• Apply ORM and database design principles (entity modelling, relationships, normalization, indexing, ORMs such as SQL Alchemy / Entity Framework where used) when designing data models and mappings.
• Ensure data quality and data contract validation are embedded within pipelines (unit tests, schema checks, monitoring) and work with stakeholders to remediate upstream issues.
• Continue existing accountabilities around data fabric/lake/warehouse design, ETL/ELT development, stakeholder engagement, documentation and promoting single-source-of-truth usage across the business.
• Provide day-to-day operational support for AI agents and associated automation capabilities.
•…