Data Engineer

Location: Bengaluru

Linked In Job Post – Data Engineer

Job Title:

Data Engineer

Experience:

4+ Years

Location:

India / Hybrid (as per client requirement)

Job Summary
We are looking for a skilled  Data Engineer  with strong experience in building and operationalizing large-scale data pipelines and ETL workflows. The ideal candidate should have hands-on expertise with  PySpark, Apache Airflow, and AWS data services  to design, develop, and maintain scalable data platforms and analytics solutions.

Key Responsibilities
Build scalable  ETL/ELT data pipelines using PySpark  on distributed processing frameworks.
Design and orchestrate workflows using  Apache Airflow  including DAG creation, scheduling, and monitoring.
Develop data ingestion and transformation jobs using  AWS Glue .
Manage secure and compliant data access using  AWS Lake Formation .
Maintain and optimize  AWS Glue Data Catalog  for schema and metadata management.
Collaborate with analytics teams to publish datasets for  BI and dashboards .
Build and support data visualizations using  Amazon Quick Sight .
Ensure data quality, reliability, and performance across all data pipelines.
Required Skills
Strong hands-on experience with  PySpark for large-scale data processing
Deep knowledge of  Apache Airflow DAGs and workflow orchestration
Expertise in  AWS Glue (ETL jobs, Glue Studio, Crawlers)

Experience with  AWS Lake Formation for data governance
Familiarity with  AWS Glue Data Catalog and metadata management
Nice to Have

Experience with  AWS S3, Athena, Redshift, or EMR
Knowledge of  Python-based automation and testing
Exposure to  Dev Ops / Infrastructure as Code (Terraform or Cloud Formation)
Top 5 Mandatory Skills (For Naukri / Linked In Search)
Py Spark
Apache Airflow
AWS Glue
AWS Lake Formation
AWS Glue Data Catalog