Data Engineer

The Lead Data Engineer will be responsible for designing, building, and optimizing scalable data pipelines. The role requires strong hands‑on experience in data engineering, distributed data processing.

• Design, build, and maintain ETL/ELT pipelines.
• Develop and optimize data ingestion frameworks, data transformations, and end‑to‑end workflows for batch and streaming use cases.
• Work with stakeholders to understand data requirements and translate them into scalable data engineering solutions.
• Manage and optimize codes
• Ensure data quality, reliability, and observability through validation frameworks and monitoring.
• Contribute to data modelling, metadata management, and best practices within the data platform.
• Collaborate closely with data scientists, analysts, and business teams to support analytics and ML workloads.

• 10+ years of experience in data engineering development
• Hands‑on experience with 5+ years in PySpark distributed data processing.
• 3+ years of indepth hands on experience in Python
• Atleast 3 + years of experience in Hadoop
• 3+ years of experience as a Data Engineer in AWS or Azure
• Solid SQL knowledge and experience working with large‑scale datasets
• Good understanding of Delta Lake, medallion architecture, and scalable lakehouse patterns.
• Good understanding of CI/CD, Git, and modern Dev Ops practices for data pipelines.
• Familiarity with structured/unstructured data, data quality frameworks, and performance tuning.