Senior Data Engineer; on-site - SC

Methods is currently recruiting for a Data Engineer to join our team on a permanent basis. This role will include some work on client sites and frequent visits to our main offices in London.

• Design, construct, and maintain efficient and reliable data pipelines using Python/Go/Azure Data Factory, supporting both streaming and batch data processing across structured, semi‑structured, and unstructured data in on‑premises and Azure environments.
• Implement and manage hybrid cloud and data storage solutions leveraging on‑premises infrastructure and Azure, ensuring seamless data integration and accessibility across platforms.
• Employ Azure Data Factory to automate data flows and manage complex workflows within hybrid environments.
• Utilise event‑driven technologies such as Kafka and NATS to handle real‑time data streams effectively.
• Manage security setups and access controls, incorporating tools like Keycloak to protect data integrity and comply with legal standards across all data platforms.
• Design and develop PostgreSQL databases, ensuring high performance and availability across diverse deployment scenarios.
• Deploy and manage scalable data operations that support advanced analytics and data‑driven decision‑making.
• Coordinate on‑site, full‑time work and adhere to security clearance requirements (Active SC with willingness to progress to DV).

• Strong Python skills for scripting and automating data processes.
• Demonstrable experience developing and optimising ETL or ELT workflows, particularly in hybrid (on‑premises and Azure) environments.
• Practical experience with Azure Data Factory for workflow automation.
• Proficiency in event streaming platforms such as Kafka and NATS.
• Experience implementing security practices and tools, including Keycloak.
• Background in managing Elasticsearch and PostgreSQL in environments spanning on‑premises and cloud infrastructures.

• Certifications in Azure and other relevant cloud or on‑premises technologies.
• Minimum of 5 years of experience in data engineering with significant exposure to managing infrastructure in both on‑premises and cloud settings.
• Some Dev Ops engineering experience.

• Autonomy to develop and grow your skills and experience.
• Be part of exciting project work that makes a difference in society.
• Strong, inspiring, and thought‑provoking leadership.
• A supportive and collaborative environment.
• Access to Linked In Learning, a management development programme, and training.
• 24/7 confidential employee assistance programme.
• Flexible working, including home working and part time.
• Office parties, breakfast Tuesdays, monthly pizza Thursdays, Thirsty Thursdays, and commitment to charitable causes.
• 25 days of annual leave a year, plus bank holidays, with the option to buy 5 extra days each year.
• 2 paid days per year to volunteer in local communities or within a charity organisation.
• Salary Exchange Scheme with 4% employer contribution and 5% employee contribution.
• Life assurance of 4 times base salary.
• Non‑contributory private medical and worldwide travel insurance (spouse and dependents included).
• Enhanced maternity and paternity pay.
• Season ticket loan and cycle to work scheme.

For a full list of benefits please visit our website ().