Lead Big Data Engineer; GCP

About the Role

In this role, you will lead a data migration stream as part of an AWS to GCP migration program. This is a hands‑on leadership role: you will coordinate technical activities, guide engineers, and actively participate in development, troubleshooting, code review, and validation. The role requires close collaboration with the Soft Serve delivery team, client stakeholders, and vendor professional services.

• Lead day‑to‑day technical execution of the data migration stream
• Coordinate implementation across Soft Serve, client, vendor, and platform teams
• Perform hands‑on development, troubleshooting, code review, and validation
• Guide engineers, clarify technical approach, and ensure implementation quality
• Collaborate with the Architect on design decisions, risks, and architecture alignment
• Collaborate with Dev Ops on CI/CD, Terraform, IAM, infrastructure, and environment setup
• Collaborate with Business Analyst on scope, dependencies, acceptance criteria, and client alignment
• Track technical risks, blockers, assumptions, and cross‑team dependencies
• Support migration planning, validation, cutover, and post‑cutover stabilization
• Ensure clear communication, transparent status, and timely escalation

• Strong hands‑on experience with Google Cloud Platform (GCP), including Cloud Composer (Apache Airflow), Dataflow, GCS, Pub/Sub, Cloud Run, Cloud Functions, Secret Manager, IAM, Cloud Logging, and Monitoring in enterprise‑scale data platforms
• Deep practical expertise in Apache Airflow: DAG development, orchestration, scheduling, retries, monitoring, alerting, dependency management, and migration to Cloud Composer
• Solid experience with relational databases, including schema design, SQL optimization, performance tuning, and migrations across Oracle, PostgreSQL, MySQL, and SQL Server
• Working knowledge of AWS services such as S3, MWAA, Lambda, Firehose, API Gateway, ECS/ECR, IAM, and AWS Secrets Manager, with ability to translate AWS solutions into GCP architectures
• Strong understanding of Terraform and CI/CD practices, including infrastructure as code, modularization, and environment management
• Practical programming skills in Python, SQL, and Bash for automation, orchestration, and data processing
• Exposure to Databricks and Snowflake for analytics, data modeling, and pipeline integration
• Strong ownership mindset with proven ability to independently lead migration streams, remove blockers, drive delivery, and collaborate effectively across engineering, Dev Ops, and stakeholder teams
• Upper‑Intermediate to Advanced English skills enabling effective technical communication and cross‑team collaboration

Soft Serve is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment regardless of race, color, religion, age, sex, nationality, disability, sexual orientation, gender identity and expression, veteran status, and other protected characteristics under applicable law. Let’s put your talents and experience in motion with Soft Serve.