Senior Data Engineer

Overview

Senior data engineers at Thoughtworks are engineers who build, maintain and test the software architecture and infrastructure for managing data applications. They are involved in developing core capabilities which include technical and functional data platforms. They are the anchor for functional streams of work and are accountable for timely delivery. They work on the latest big data tools, frameworks and offerings (data mesh, etc.),

while also being involved in enabling credible and collaborative problem solving to execute on a strategy.

• You will develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions.
• You will develop intricate data processing pipelines, addressing clients’ most challenging problems.
• You will collaborate with data scientists to design scalable implementations of their models.
• You will write clean, iterative code using TDD and leverage various continuous delivery practices to deploy, support and operate data pipelines.
• You will use different distributed storage and computing technologies from the plethora of options available.
• You will develop data models by selecting from a variety of modeling techniques and implementing the chosen data model using the appropriate technology stack.
• You will collaborate with the team on the areas of data governance, data security and data privacy.
• You will incorporate data quality into your day-to-day work.

• Technical Skills
• Working with data excites you: you can build and operate data pipelines, and maintain data storage, all within distributed systems.
• You have hands-on experience of data modeling and modern data engineering tools and platforms.
• You have experience in writing clean, high-quality code using the preferred programming language.
• You have built and deployed large-scale data pipelines and data-centric applications using any of the distributed storage platforms and distributed processing platforms in a production setting.
• You have experience with distributed storage and processing platforms in a production setting.