Data Engineer

Location: Bengaluru

AWS Data Engineer

– Job Description

Mandatory

Role: AWS Data Engineer

Required Technical Skill Set:
Python, PySpark, AWS Glue, AWS Lambda, Amazon S3, Redshift, Athena, EMR, DynamoDB, Kinesis, IAM, CI/CD, Databricks

Desired Experience Range: 6–8 Years

Location of Requirement:
Pan India

Desired Competencies

Must-Have

- Strong proficiency in Python and PySpark for large‑scale distributed data processing.
- Experience building data ingestion pipelines using AWS Glue, Kinesis, Lambda, API-based ingestion.

- Experience with Glue ETL, Spark on EMR, or Databricks on AWS.
- Expertise with Amazon S3, Redshift, DynamoDB, Athena.
- Strong understanding of ETL/ELT, dimensional modeling, data lakes, and warehouse architecture.

Good-to-Have

- Experience with Dev Ops & Automation using Git, AWS Code Pipeline, Cloud Watch, CI/CD.
- Performance tuning of Spark jobs, Glue jobs, and Redshift queries.
- Knowledge of data governance, quality checks, and cataloging using Glue Catalog or Lake Formation.

Key Responsibilities

- Design, develop, and maintain ETL/ELT pipelines using AWS Glue, Lambda, Spark, and Databricks.
- Build scalable data lake and data warehouse solutions using S3, Redshift, and Athena.
- Orchestrate workflows using Glue Workflows, Step Functions, Airflow, or Databricks Workflows.
- Integrate AWS services such as S3, Glue, Lambda, Redshift, Kinesis, and DynamoDB.
- Optimize pipelines for performance, reliability, and cost.
- Implement data quality checks, cataloging, lineage, and security best practices.
- Collaborate with business and analytics teams to translate data requirements.

er