Data Engineer
-
Software Development
Job Description & How to Apply Below
- Data Engineer - 24 Month Fixed Term Contract
You will work across the data platform that supports operational services, integrations, reporting and decision-making across Millbrook. This is not a role where strong SQL can be replaced by knowing the latest cloud product. We have a large SQL Server estate with years of business logic in stored procedures, views, functions and integrations, and we need somebody who can understand and improve that safely while helping us move toward a modern Microsoft data platform.
The direction is Microsoft Fabric, governed data products and a clearer bronze/silver/gold architecture, but getting there depends on properly understanding the estate we have today.
The estate you are joiningThe current data estate is heavily SQL Server based. A lot of important operational and reporting logic lives in large stored procedures, views, functions, scheduled jobs and established ETL processes. Some of that SQL is complex, old and tightly coupled to business processes, so being able to read, diagnose and safely change it is a core part of the job.
Alongside that we have Power BI, on-premises gateways, Azure services and integration flows that need to keep working while we modernise. The destination is a much clearer Microsoft Fabric platform with governed data products, bronze/silver/gold layers and less dependence on opaque stored-procedure logic, but we need to get there without losing the business knowledge embedded in the current estate.
What you will be responsible for SQL and the esisting data estate- Build, support and improve complex SQL Server solutions used by operational systems, reporting and integrations.
- Work confidently with large stored procedures, views, functions, dynamic SQL, temp tables, table variables and established T-SQL patterns.
- Read and understand stored procedures that may contain a significant amount of business logic before changing or replacing them.
- Troubleshoot slow or unreliable SQL using execution plans, indexing, statistics, blocking/waits and query-level diagnostics.
- Refactor high-risk or difficult SQL pragmatically, preserving business behaviour while improving maintainability, performance and testability.
- Support data migrations, reconciliations, extracts and operational data fixes where correctness and traceability matter.
- Understand dependencies between operational databases, replicas, reporting workloads, integrations and downstream consumers before making changes.
- Design and maintain reliable ETL/ELT pipelines that ingest, transform and publish data from operational and third-party systems.
- Combine data from multiple systems into trusted reusable datasets for reporting, analytics and operational processes.
- Build and support integrations using SQL, APIs, files and appropriate Azure data or messaging services.
- Design transformations that are testable, repeatable and observable rather than hidden in manual processes.
- Work closely with Software Engineering where data movement is part of an application or integration flow.
- Handle schema changes, late-arriving data, retries, reconciliation and error handling as normal engineering concerns.
- Build validation and reconciliation into pipelines so problems are identified early and traced back to source.
- Investigate data-quality issues and work with system owners to fix root causes, not just patch reports.
- Work with BI colleagues to provide well-structured data for Power BI and semantic models.
- Understand the effect that model, query and refresh design has on report performance, SQL workload and gateway load.
- Design dimensional or analytical models where appropriate, with clear grain, keys and business definitions.
- Document important data definitions, lineage, ownership and transformation logic.
- Use Git, peer review and CI/CD for database and data-platform changes wherever practical.
- Automate deployment, validation and monitoring rather than relying on manual scripts and undocumented steps.
- Apply appropriate security, access, retention and privacy controls to healthcare and business data.
- Support incidents and production issues with clear root-cause analysis and follow-through.
- Treat data pipelines and SQL code as production software, with testing, observability and controlled change.
- Microsoft SQL Server and very strong T-SQL, including large and complex stored procedures, dynamic SQL, views, functions, scheduled/database-driven…
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction