Senior GCP Data Engineer

Location: Bengaluru

Job Title Senior GCP Data Engineer
Total IT Experience (in Yrs.) 6-10
Relevant Experience Required
(in Yrs.)
· Proven experience in designing, architecting, and implementing large-scale data solutions on Google Cloud Platform (GCP).
· Expertise in data ingestion, transformation, orchestration, and analytics using native GCP services.
· Hands-on experience in data migration, modernization, and data lakehouse architecture.
· Strong experience leading data engineering teams, reviewing code, and ensuring best practices.
· Deep understanding of data governance, cost optimization, and security within GCP.
· Ability to work closely with data architects, business stakeholders, and cross-functional teams to define enterprise data strategies.
· Experience in end-to-end lifecycle of data engineering projects using Agile and Dev Ops frameworks.
Language Requirement English
Key words to search in resume Senior GCP Data Engineer, Google Cloud, Big Query, Dataflow, Dataproc, Cloud Composer, Pub/Sub, Cloud Storage, Python, SQL, Data Lake, Data Warehouse, ETL, Airflow, Terraform, Dev Ops, GKE, Cloud Run, Architecture, Leadership, Data Modernization, CI/CD
Technical/Functional Skills
- MUST HAVE SKILLS
· Expert-level knowledge of Google Cloud Platform services including:
· Big Query – advanced SQL, partitioning, optimization, and cost control
· Dataflow (Apache Beam) – large-scale batch and streaming data pipelines
· Dataproc – Spark and Hadoop-based distributed processing
· Cloud Composer (Airflow) – workflow orchestration, scheduling, and dependency management
· Pub/Sub – event-driven, real-time ingestion
· Cloud Storage (GCS) – data lake setup, lifecycle management, access control
· Strong proficiency in Python (Pandas, PySpark, Beam SDK) and SQL for data wrangling and transformation.
· Expertise in ETL/ELT design, data lake/data warehouse/lakehouse architecture.
· Experience with Dev Ops & CI/CD using tools such as Git Hub Actions, Jenkins, or Cloud Build.
· Hands-on with Terraform or Deployment Manager for Infrastructure as Code (IaC).
· Solid understanding of GCP IAM, VPC, security policies, and encryption mechanisms.
· Familiarity with data quality, metadata, lineage, and cataloging tools (e.g., Data Catalog).
· Proficiency in cost governance, performance tuning, and resource monitoring (Cloud Monitoring/Logging).

· Knowledge of modern data frameworks and data integration between systems like Kafka, Snowflake, or Looker.
Secondary Skills
· Understanding of machine learning workflows using Big Query ML, Vertex AI, or Tensor Flow.
· Familiarity with Docker, Kubernetes (GKE), and Cloud Run for containerized workloads.
· Experience with data governance frameworks and compliance standards (GDPR, HIPAA).
· Working knowledge of BI/analytics tools such as Looker, Power BI, or Tableau.
· Strong documentation, mentoring, and stakeholder communication skills.
· Exposure to multi-cloud environments (AWS, Azure) for data integration.
Responsibilities
· Lead the design and implementation of scalable data pipelines using GCP native tools (Dataflow, Composer, Big Query, Dataproc).
· Architect and develop data lakehouse and warehouse solutions that meet business analytics and AI/ML requirements.
· Manage and mentor junior data engineers, ensuring high-quality and reusable code.
· Collaborate with data architects to align on data strategy, security, and best practices.
· Oversee data ingestion from diverse sources (on-prem, APIs, Kafka, databases) into GCP.
· Implement ETL/ELT processes, optimize for performance, and ensure data quality and reliability.
· Apply infrastructure automation using Terraform and CI/CD for seamless deployments.
· Define and enforce governance, lineage, and metadata standards.
· Monitor workloads, optimize Big Query costs, and troubleshoot pipeline performance issues.
· Drive innovation and proof-of-concepts for modernizing enterprise data platforms.
· Present technical designs and architecture recommendations to senior stakeholders.
· Collaborate across teams (security, analytics, ML) to deliver end-to-end data-driven solutions.
· Ensure all data systems comply with organizational security, compliance, and DR policies.