Senior Data Engineer

Responsibilities

• Design and develop scalable data pipelines using AWS services such as S3, Redshift, and Lambda.
• Implement real-time data processing solutions using Apache Kafka to handle streaming data efficiently.
• Utilize Python and PySpark for data transformation and analysis to ensure high-quality deliverables.
• Collaborate with data scientists and analysts to understand data requirements and provide appropriate data solutions.
• Conduct data modeling and data warehouse optimization to enhance query performance and support analytics needs.
• Leverage dbt for data transformation workflows and ensure version control of analytics code.
• Participate in architecture discussions to design modern data infrastructure on Azure that supports future growth and scalability.

• Bachelor s degree in Computer Science, Engineering, or a related field.
• Minimum of 8 years of experience in data engineering or a related field.
• Strong proficiency in AWS services and architecture for data processing and storage.
• Experience with Apache Kafka and its ecosystem for handling real-time data streams.
• Proficient in Python and PySpark with hands-on experience in building data processing applications.
• Familiarity with dbt for data transformation and analytics workflow management.
• Knowledge of Azure cloud services and how they can be integrated into data solutions.