Digital Solution Architect

Job Title Digital Solution Architect About the Role Job purpose

The Data Engineer is accountable for the design, implementation and release from an IT perspective of modern data applications and pipelines of running projects as well as new initiatives in the context of the Digital Lab.

The professional will be responsible to build integrations with external data systems, to enable data collection from a wide variety of data sources, to transform and combine properly these data in order to enable Advanced Data Analytics use cases.

The professional will also be responsible to deploy, manage and maintain the data platform in the Cloud that is the backbone for any data related project of the organization.

The professional will work closely with Business and R&D departments as IT function will orchestrate and facilitate Cloud, IoT and Big Data projects following the Enterprise Architect guidance and collaborate with others IT Enterprise functions. The professional ensures compliance with advised BSEMIA data management rules, security policies, tools, and technology stack. The professional has experience with Agile frameworks and Strategic mind set with hands-on mentality, creative / out-of-the-box, flexible / courageous, showing ownership and agility.

Responsibilities:

Design, develop and deliver reliable modern data pipeline to support any data related projects

Implement Batch processes for data Ingestion, Processing and Transformations Implement Streaming processes for data Ingestion, Processing and Transformations Serve and provide access to the final datasets to Data Science Team to enable Advanced Analytics Manage data and related metadata in Big Data Warehouse, Structured and/or Unstructured Data stores

Design, Administer and Maintain Data Lake for Big Data Analytics Design, Develop and Deliver components for Data Integration and Extraction Operates data pipelines and cloud resources in case of failures and bugfixes

Deployment, configuration and monitoring of the Cloud Data Platform infrastructure and services

Works closely with Data Architect and Data Scientists Support ML and Predictive Models deployment in the cloud

Facilitates and orchestrates Cloud, IoT, Analytics use cases.

Works in Agile teams to implement new PoC and MVP projects, prepares and runs demos as appropriate.

Familiarities with Web Development technologies and best practices, with special reference to the Microsoft world.

Qualifications & Experience Required Education, Master, other certification:

Master’s Degree in Computer Science or equivalent.

Experience (years):

Minimum 3 years of experience working on Big Data projects, using technologies like HDFS, Hadoop, Spark, Spark Streaming, Hive, HBase …Minimum 3 years of experience developing Batch and Streaming data pipelines especially in Microsoft Azure and using Spark.

Technical skills:

Advanced experience with IaaS and PaaS in Azure especially using Azure data services like Azure Data Factories, Azure Databricks, Azure Data Lake Store, Azure SQL database, Azure HDInsight. AWS knowledge is a nice to have.

Broad understanding and experience with software development best practices, cloud oriented architectures and OLTP/OLAP workloads.

Advanced knowledge of SQL language and Spark for Big data transformation is a must.

Knowledge and experience in Serverless computing components development and with services like Azure Functions or AWS Lambda.

Strong understanding of data modeling, algorithms, and data transformation techniques and BI tools

Strong understanding of Big Data architecture and Data transformations patterns

Strong understanding of relational DB and Data Warehouses (such as SQL Server, Oracle, MySQL, Postgres or similar and/or Azure SQL database service)
Good understanding of…