Senior Data Engineer On site

Excellent opportunity to work on site in Abu Dhabi.

Contract -6-12 Months, extendable
Employment Type: Full-Time
Benefits:
Health insurance, Medical insurance, flight tickets etc will be taken care by the hiring company

• Design, develop, and maintain scalable data pipelines, ETL/ELT processes, data models, and schemas to process large-scale datasets for analytics and machine learning.
• Build and optimize high-performance data processing systems using technologies such as Apache Spark, Delta Lake, Kafka, and Hadoop
  .
• Develop real-time data pipelines using Apache Kafka
  , Azure Event Hub
  , or similar streaming technologies.
• Design and implement cloud-native data solutions on Azure ensuring scalability, reliability, performance, and cost optimization.
• Build and maintain data lakes, data warehouses, and lakehouse architectures, including data modeling, ETL processes, and data quality management.
• Implement data validation, monitoring, and testing frameworks using tools such as Great Expectations
  .
• Ensure data governance, security, access control, and compliance with industry standards for handling sensitive data, including Protected Health Information (PHI).
• Support healthcare data interoperability standards such as FHIR
  .
• Collaborate with Data Scientists, Data Analysts, Software Engineers, and Product teams to deliver reliable, high-quality data solutions.
• Continuously evaluate and adopt modern data engineering tools, technologies, and best practices.

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.
• 5+ years of experience building and managing production-grade data engineering platforms and pipelines.
• Strong programming skills in Python and SQL
  .
• Hands-on experience with:
• Apache Spark
• Apache Kafka
• Delta Lake
• Hadoop ecosystem
• Presto/Trino
• Workflow orchestration tools (Airflow or similar)
• Docker and Kubernetes
• Experience designing scalable ETL/ELT pipelines, data models, and distributed data systems.
• Experience with cloud platforms such as Microsoft Azure
• Strong knowledge of data lakes, data warehouses, and lakehouse architecture.
• Experience with data governance, Master Data Management (MDM), data cataloging, data anonymization, and pseudonymization.
• Strong analytical, problem-solving, and communication skills.
• Ability to work effectively in a collaborative Agile environment.