Senior Data Engineer

Location: Durbanville

Description

aculocity
Aculocity Data Team:
Senior Data Engineer, Job description

Premise:

The Senior Data Engineer is a lead resource in building and developing the Aculocity Data capabilities.

As a Senior Data Engineer, you are at the heart of our agile Data Engineering team, spearheading the creation, innovation, and optimization of cutting-edge data infrastructure and pipelines. Your leadership and technical prowess will empower us to deliver game-changing reporting, analytics, and machine learning solutions that propel our business forward. Harnessing your deep experience, you’ll not only architect and nurture mission-critical data systems but also energize and mentor junior engineers, ensuring that complex business needs are transformed into dynamic, scalable data solutions.

Get ready to make your mark and drive real impact in a fast-paced, high-growth environment!

Data Engineer

Job Responsibilities:

Serve as the subject matter expert for data and systems.

Develops and maintains scalable data pipelines and build new API integrations to support continuing increases in data volume and complexity.

Collaborates with analytics and business teams to improve data models that feed business

intelligence tools, increase data accessibility and foster data-driven decision making across the organization.

Implements processes and systems to monitor data quality, ensuring production data is

always accurate and available for key stakeholders and business processes that depend on it.

Develop end-to-end Data pipelines encompassing the full lifecycle from data ingestion,

data transformation and Data Marts in our Enterprise Data Warehouse.

Collaborate closely with Date Scientists to accelerate development of ML and AI solutions ready for production.

Setup CI/CD/CT pipelines

Contributes to engineering wiki, and documents work.

Performs data analysis required to troubleshoot data related issues and assists in the resolution of data issues.

Works closely with cross-functional teams of frontend and backend engineers, product

managers, and analysts to enhance data models and support advanced BI and analytics.

Defines company data assets (data models), ETL jobs to populate data models.

Designs data integrations and data quality frameworks.

Designs and evaluates open source and vendor tools for data lineage.

Works closely with all business units and engineering teams to develop strategy for long term data platform architecture.

Mentor junior data engineers, lead code reviews, and promote best practices and skill

development.

Data Engineer Qualifications /

Skills:

Knowledge of best practices and IT operations in an always-up, always-available service.

Establish and promote best practices for data pipeline and data model design and development.

Experience with or knowledge of Agile Software Development methodologies.

Excellent problem solving, creativity, attention to detail and troubleshooting skills.

Process oriented with great documentation skills.

Excellent oral and written communication skills with a keen sense of customer service.

Eagerness to learn and upskill to stay at the forefront of technology offerings in the market.

Team player mindset with an enthusiasm for collaboration and technology

Education, Experience, and Licensing Requirements:

Must Have:

BSc or MSc degree in Computer Science or a related technical field

5+ years of Python development experience

5+ years of MS SQL experience (Postgre

SQL experience is a plus)

5+ years of experience with Warehouse Architecture, schema design and dimensional data modeling.

Ability in managing and communicating data warehouse plans to internal clients

Experience designing, building, and maintaining data processing systems on multiple platforms both Cloud (Azure, AWS, MS Fabric) and On-Premises (MS SQL Server, SSIS) is advantageous

Good experience in Apache Spark.

Experience debugging and reasoning about production issues is essential.

Experience presenting demos and training of technical, non-technical and analytical resources

Advantages to have:

Experience in Data Analytics and Business Intelligence tools such as Power BI

Experience in data streaming is advantageous . Kafka

IoT device and systems integration