Data Engineer

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
• Build and optimize large-scale data processing solutions using Python, PySpark, and Java.
• Develop data ingestion frameworks for structured and unstructured data sources.
• Integrate data from various systems through REST APIs, streaming platforms, and batch processing.
• Work with large datasets in distributed computing environments.
• Develop and support real-time and batch data processing solutions using Kafka and Hadoop ecosystem technologies.
• Implement scalable streaming data pipelines and event-driven architectures.
• Monitor and optimize data workflows for performance and reliability.
• Write complex SQL queries for data extraction, transformation, and analysis.
• Design and optimize database schemas and data models.
• Ensure data quality, consistency, and governance standards are maintained.
• Participate in Agile ceremonies including sprint planning, daily stand-ups, backlog grooming, and retrospectives.
• Use Jira for project tracking, issue management, and sprint execution.
• Collaborate with Data Architects, Data Scientists, Business Analysts, and Application Development teams.
• Work independently as well as within a collaborative team environment.
• Troubleshoot and resolve data-related issues across environments.
• Perform root cause analysis and implement long-term solutions.
• Support production deployments and ongoing maintenance activities.

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 4+ years of experience in Data Engineering or Big Data development.
• Hands-on experience with Python, PySpark, Java, Kafka, Hadoop, REST APIs, and SQL.
• Strong analytical and problem-solving abilities.
• Ability to work effectively in both team-oriented and independent environments.