Data Software Engineer

We are a trusted, specialised consultancy with a mission to bring expert technology leadership to businesses at every stage of their growth journey. Our work spans multiple clients and providers, helping them deliver technology initiatives with precision, strategic alignment, and lasting value. Operating at the intersection of technical excellence and business strategy, we ensure projects not only launch successfully but also drive broader organisational goals.

As part of our ongoing support for a key client, we're seeking a dynamic, strategically minded Data Software Engineer who can bring strong software engineering fundamentals to building the services and capabilities of our data platform designing and developing the systems that orchestrate, configure and expose ingestion, transformation and data-quality capabilities as a service, with a full Dev Ops mindset.

We are looking for someone who is comfortable building software for production: not only delivering new functionality, but also making sure what we build is reliable, monitored, maintainable and easy to support afterwards.

• Build the services that orchestrate and expose ingestion, transformation and data-quality capabilities as a service to other teams.
• Develop ingestion and connector‑based integration capabilities using Kafka and Kafka Connect.
• Integrate and orchestrate distributed processing engines (e.g. Apache Spark, Flink or Beam) behind clean service APIs, abstracting the underlying execution.
• Implement comprehensive monitoring and alerting to track service health, data quality and resource utilisation.
• Investigate and support production issues, helping to improve reliability and reduce recurring operational pain.
• Support performance‑tuning initiatives and find new ways to increase efficiency and throughput.
• Use approved AI‑assisted engineering tools responsibly to support development, testing, documentation and troubleshooting.
• Direct AI coding agents as part of everyday delivery --- providing clear context, reviewing output critically, and owning the quality, security and design of what ships.
• Work in a team of engineers and product owners using agile methodologies, owning your work end‑to‑end under a Dev Ops philosophy across the full SDLC.
• Follow best practices across design, coding, testing, operations and monitoring, and give and receive constructive feedback through code reviews.
• Collaborate, peer review, cross‑skill and share expertise with other team members.
• Contribute to continuous improvement, suggesting and implementing enhancements to optimise the SDLC.
• Stay current with advancements in software, data and streaming technologies, exploring new tools and techniques.
• Design, develop, test, deploy, maintain and support the platform services and capabilities of our data platform.

• Solid software‑engineering fundamentals:
  Object‑Oriented Programming, strong back‑end skills (e.g. multithreading, OOP) and relevant experience in Java and/or Python.
• Expertise in crafting, evaluating, rolling out, maintaining and evolving software solutions and services.
• Experience working with high‑velocity, high‑volume data across relational, semi‑structured and unstructured formats.
• Expertise with data ingestion using Kafka and Kafka Connect.
• Hands‑on experience with at least one stream/batch processing technology --- Apache Spark, Flink or Beam (Confluent Platform a plus).
• Strong SQL skills, including query writing and database tuning, with experience in relational and/or No
  
  SQL database systems.
• Understanding of data modelling, architecture and how other systems work.
• An agile approach to development and a CI/CD mindset, with understanding of CI/CD tooling.
• A pursuit of quality through comprehensive unit and integration testing.
• A pragmatic approach to building software that is reliable, observable and maintainable in production.
• Familiarity with AI‑generated code and how to assess it --- applying the same considerations of testing, readability and architectural fit as any other contribution, and an awareness of when AI assistance adds value across the SDLC.
• Problem‑solving skills, willingness to take ownership, and enthusiasm in…