GCP Senior Data Engineer

GCP Senior Data Engineer | 6-10 Years of experience |
Chennai/ Hyderabad / Kolkata /Bangalore

Required Technical Skills
Python
SQL
Py Spark
GCP (Preferred) or Azure
Snowflake
Airflow
Jenkins / Jira / Git
CI/CD
Kubernetes / Docker

Must-Have
Strong development experience in Python, PySpark, and SQL.

Experience with cloud platforms (GCP preferred).
Experience working in an Agile delivery environment.

Experience with orchestration tools such as Airflow and ADF.

Experience with GCP services:
GCS Bucket
Cloud Functions
Dataproc
Dataflow
Pub/Sub
Big Query
Hands-on experience in Snowflake.
Experience building automated data pipelines.
Experience in data analysis and data exploration.
Experience working in a multi-developer environment using version control tools such as Git.
Strong critical thinking, communication, and problem-solving skills.

Good-to-Have
Understanding of Dev Ops best practices and CI/CD.
Understanding of containerization technologies such as Kubernetes and Docker.
Healthcare domain knowledge.

Key Responsibilities
Perform data analytics on large-scale datasets to solve complex business problems and recommend solutions.
Develop and enhance end-to-end data pipelines using big data technologies such as Hadoop, Cloud platforms, and the Spark ecosystem.
Troubleshoot complex data structures and perform root cause analysis to proactively resolve operational and product issues.
Develop and document test cases, execute testing, and share UAT results as part of Agile sprints.
Perform end-to-end data pipeline integration testing in compliance with change management processes and standards.
Adhere to coding, design, and application standards, including Python and SQL best practices.
Leverage Dev Ops practices for production deployments, including CI/CD, test-driven development, and version control using Jenkins, Jira, and Git.
Collaborate with business stakeholders to define product objectives, business requirements, system designs, and key performance indicators (KPIs).