Data Engineer

Overview:
We are seeking an experienced Engineer with strong programming and cloud platform skills. The ideal candidate will have hands-on experience with a variety of AWS services, a solid background in Python development, SQL, and data modeling, and the ability to leverage AI tools such as Git Hub Copilot to enhance productivity.

Key Responsibilities:

• Write efficient, maintainable code in Python/Pyspark, leveraging frameworks like PySpark for data processing.
• Investigate and troubleshoot data pipelines using PySpark, with a strong understanding of data structures and processing.
• Develop and optimize SQL queries; understand and implement Slowly Changing Dimensions (SCD) Type 1 and Type 2 in data warehousing.
• Design, develop, and deploy APIs on AWS, ensuring scalability and reliability.
• Work extensively with AWS services such as ECS, Lambda, SNS, and DynamoDB.
• Build, test, and deploy applications using CI/CD pipelines, ensuring smooth and automated delivery.
• Utilize AI-powered development tools (e.g., Git Hub Copilot) to accelerate coding and improve code quality.
• Collaborate with cross-functional teams to deliver robust cloud solutions.
• Troubleshoot and resolve issues related to cloud infrastructure and application performance.

Required Skills:

• Strong knowledge of the AWS platform and its core services (ECS, Lambda, SNS, DynamoDB).
• Proficient in Python programming; experience with PySpark is highly desirable.
• Solid understanding of SQL and data modeling concepts, including SCD Type 1 and Type 2.
• Experience in building and deploying APIs (FastAPI/REST/RESTful/Flask) on AWS /(ECS).
• Good understanding of CI/CD processes and tools.
• Ability to work with AI-powered development tools such as Git Hub Copilot.