Senior Data Engineer

At Canopius, our delivery teams are responsible for ensuring that business users can effectively harness data insights to drive strategic decision making. Our data strategy is centred around an enterprise Lakehouse platform on Databricks, avoiding fragmented, ungoverned silos on legacy technologies that hamper creativity and scalability. We are building a governed, interoperable data estate that enables self-service for our business teams and provides the trusted foundation for advanced analytics, machine learning and AI to accelerate change across our industry.

This role is an opportunity to apply yourexpertisein Azure technologies, data storage and data engineering techniques to build, extend andmaintainour data ecosystem, which is central to all reporting and analytics across Canopius. You will contribute to key business transformation projects, working closely with senior business stakeholders and leveraging modern technologies. The role includes line management responsibility for 2 junior and a mid-level data engineer, with the potential to grow over time as required.

As a senior member of the team, you will also act as a role model and mentor to other engineers, providing guidance and support.

The ideal candidate is an experienced data engineer looking for a new challenge who is enthusiastic about using technology to accelerate change within our industry. You should have the ability to understand business problems and deliver efficient and reliable solutions tailored to our unique challenges, while keeping to a high degree of technical excellence and good data engineering practices. You should be comfortable collaborating and working as part of a dynamic multi-disciplinary team.

This role is central to delivering Canopius’ data strategy through development of new data capabilities and supporting the final transition from legacy SQL Server-based environments. Alongside hands-on delivery, the role provides technical leadership across data engineering initiatives, helping to establish a future-ready, cloud-native data ecosystem.

This is a hybrid role, working 2-3 days per week from our Manchester office.

Responsibilities will include:

• Provide technical leadership and direction across data engineering initiatives.
• Design solutions for production operation, including documentation, monitoring, alerting, logging, access and defined support responsibilities. Ensure solutions are secure, scalable, maintainable, and aligned with organisational standards and governance requirements.
• Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture.
• Implement data validation and reconciliation checks, and automated testing across pipelines to ensure trusted, accurate outputs.
• Support the migration and re-platforming of legacy SQL Server data warehouse processes as required.
• Challenge requirements constructively and provide guidance on what is technically feasible, scalable, and commercially viable.
• Communicate technical solutions, trade-offs, risks, and opportunities to both technical and non-technical audiences, managing expectations and influencing decisions with evidence-based recommendations.
• Lead technical planning activities, ensuring work is appropriately defined, estimated, and prioritised. Provide delivery estimates and communicate progress and blockers to stakeholders.
• Collaborate with Product Owners, Business Analysts, and Solution Architects to define product vision and technical roadmaps.
• Drive adoption of modern engineering practices, AI technologies, and automation opportunities across the team.
• Drive delivery of complex, end-to-end work items, seeking clarification of ambiguous specifications and proactively…