Data Platform Tech Lead

Our Data Platform Team The mission of the Data Platform team is to empower Free Agent’s engineers to reliably, securely, and sustainably derive value from our data assets.

Department: Engineering

Employment Type: Permanent - Full Time

Location: Edinburgh

Reporting To: Dave Evans

Compensation: £59,000 - £87,000 / year

Description

Our Data Platform Team is responsible for enabling the engineering organization to work with data effectively. We are looking for a Tech Lead who will participate in the design and implementation of our data platform, lead a small team of engineers, and collaborate with Data Science and Analytics Engineering to provide the right tools and processes for stakeholders to turn data into insights.

This is a team leadership role with a focus on technical contribution.

Past projects include the design and delivery of an event system processing over 100 million events per month, automating metadata management in our data lake, and implementing mechanisms for regular data quality checks. Our platform supports machine learning based automations for document extraction, bill, expense and bank transaction categorisation. We categorise over two million bank transactions every month with accounting-grade accuracy.

We are migrating to a new orchestration tool, which presents opportunities to shape Free Agent’s data tech stack and apply modern engineering practices to data pipelines.

Day to day your role will involve

• Helping us realise our vision for the next generation data platform to collect, process, store and analyse customer and business data
• Leading an agile development process for your team
• Collaborating cross-functionally with other engineering teams
• Assisting in producing and upholding common development and architectural standards across engineering teams
• Contributing code, reviewing pull requests
• Line managing your team, onboarding, coaching and mentoring new hires
• Keeping up to date with developments in data, ML and AI technology
• Working with senior engineering leadership to understand the big picture and drive the data platform to meet Free Agent’s future goals

Our data platform is built in AWS. If you have experience with equivalent tools, we encourage you to apply.

Ideally you’ll have the following skills and experience:

Data Engineering

• Hands-on experience with a modern data pipeline stack. We are migrating to Dagster, dbt and dlt, and work with structured and unstructured data from multiple sources as well as internal event data
• Experience designing and implementing data lakehouse storage architectures using tools such as S3, Redshift, Glue and Athena
• Ability to apply appropriate data modelling techniques; we use dimensional models for business reporting
• Up-to-date knowledge of data management and retention regulations, security and ethics

Software development

• Experience following a modern software development methodology to implement and test changes to complex systems. We use Python, Ruby and Git/Git Hub

Cloud infrastructure

• Experience building and managing infrastructure as code (Terraform)
• Experience deploying containerised services with Docker and Git Hub Actions
• Experience with machine learning platforms such as Sage Maker

Leadership and management

• Understanding of lean and agile development methodologies
• Experience of or enthusiasm to develop people management skills
• Keeping up to date with new techniques, technologies and resources

Please also ensure you are familiar with the hiring process for this role so your application contains all the information we need.

Our work life

We value work-life balance and a 35-hour working week. Our Edinburgh HQ is designed for collaboration with high-quality A/V equipment to support distributed teams. We operate a no-blame culture and promote a mindset of working hard, being kind to people, and letting the rest fall into place.

Work pattern for this role

This is a permanent, full time role (Monday to Friday, 9am to 5pm). The role will be based in our Edinburgh office, with the opportunity for hybrid working (2 days in the office, 3 days at home per week).

Perks

• Senior level: £59,000 - £72,000 per annum, with room to progress to £80,000 within the band
• Staff…