Data Engineer

Location: Bengaluru

AWS Data Engineer (Python, PySpark)

Experience:

5+ Years

Location:

PAN India

Employment Type:

Full-time

Job Summary
We are looking for a highly skilled  AWS Data Engineer  with strong expertise in  Python and PySpark  to design, build, and optimize scalable data pipelines on the cloud. The ideal candidate should have hands-on experience in AWS services and big data technologies, with a strong understanding of data architecture and processing frameworks.

Key Responsibilities
Design, develop, and maintain scalable  data pipelines using PySpark and Python
Work with  AWS cloud services  such as S3, Glue, Redshift, EMR, Lambda, and Athena
Perform  data ingestion, transformation, and processing  for large datasets
Optimize ETL/ELT workflows for performance and scalability
Collaborate with data scientists, analysts, and stakeholders to deliver data solutions
Implement data quality, validation, and governance practices
Monitor and troubleshoot data pipelines and workflows
Ensure best practices for  cloud security, cost optimization, and performance tuning
Required Skills
Strong experience in  Python programming
Hands-on expertise in  PySpark and Spark framework
Minimum 5+ years of experience in  data engineering

Experience with  AWS services :
S3
AWS Glue
EMR
Redshift
Lambda
Athena
Knowledge of  SQL and No

SQL databases
Experience in  ETL/ELT pipeline development
Familiarity with  data warehousing concepts
Understanding of  distributed computing systems
Preferred Skills

Experience with  Airflow or other workflow orchestration tools
Knowledge of  Kafka or real-time data streaming

Experience with  CI/CD pipelines and Dev Ops practices
Understanding of  Data Lake architecture (Lake Formation)
Exposure to  Databricks (nice to have)