DATA Engineer; Data Science & Big Data Analytics

EURECAT - Technology Center of Cataloniais seeking a highly motivated DATA ENGINEERfor its Data Science & Big Data Analytics Technological Unit.

FUNCTIONS AND RESPONSIBILITIES OF THE JOB:

· Design and deploy data pipelines for bringing data and control metadata lineage from different source systems to a data warehousing system, including queue management and real-time data processing.

· Architect reusable software practices using state of the art orchestration systems such as Airflow or Dagster and containerization stack such as Docker or Kubernetes.

· Deploy data sharing and cataloguing existing software tools to enable Data Spaces through standard building blocks from IDSA, Gaia-X and Fiware.

· Contribute to Machine Learning projects by adopting technologies for data storing, serving and versioning capabilities from blob-Storage to traditional SQL solutions and No

SQL.

· Assist the unit in multi-cloud deployments (Amazon Web Services, Azure and Google Cloud Platform).

· When applicable, deploy and design Big Data Architectures including batch and streaming paradigms such as Flink, Spark Structured Streaming, Kafka, and the Hadoop Ecosystem.

· Project management and technical lead in EU funded and private projects. Writing proposals for Horizon Europe calls (excellence, task description…)