Job Description & How to Apply Below
GCP Data Engineer
- Data Integration & system Engineer (GCP/Python/Bigquery)
Primary purpose:
Design, build, and operate scalable data pipelines and data flows that enable NBA decisioning to operate reliably across on-premise and cloud platforms. Ensure that data is correctly migrated from Hadoop/Cloudera to Google Cloud Platform and integrated with Pega CDH / Infinity to support real-time and batch decisioning for MVP and future scale.
Responsibilities
Design and implement end-to-end data pipelines across on-premise and Google Cloud Platform environments
Build and maintain batch and near real-time data flows ensuring availability, freshness, and performance
Drive migration and modernization of data from Hadoop/Cloudera to GCP (Big Query and related services)
Ensure reliable, scalable, and performant data delivery for decisioning and analytics use cases
Implement and maintain data integration with Pega CDH / Infinity, ensuring data is correctly structured and available for decision strategies and AI models
Ensure data quality, lineage, traceability, and compliance with governance and regulatory requirements
Implement monitoring, validation, and reconciliation processes for data pipelines
Collaborate with Data Scientists, Decisioning SMEs, Architects, and Product teams to align data flows with business needs
Apply modern development practices including CI/CD, version control, and automated testing
Leverage AI-assisted development tools (e.g. Git Hub Copilot, Codex, Claude, Google AI tools) to improve productivity and quality
Google Cloud Platform:
Hands-on experience with Big Query and cloud-based data pipelines (must have)
Hadoop/Cloudera ecosystem:
Strong experience with on-premise big data platforms and migration to cloud (must have)
Data pipeline development:
Proven experience building scalable batch and real-time data flows
Spark & Scala:
Strong hands-on experience with distributed data processing
Python development:
Solid programming skills for data engineering use cases
SQL expertise:
Advanced skills in data querying, transformation, and optimization
Data modeling:
Strong experience designing robust and scalable data structures
Data integration:
Experience integrating data with enterprise platforms, including Pega CDH / Infinity
Software engineering practices:
Experience with CI/CD, version control (e.g. Git), and testing frameworks
Distributed systems:
Experience working with large-scale, high-volume data environments
Data quality & governance:
Experience with validation, monitoring, lineage, and compliance requirements
AI-assisted development:
Practical experience with tools such as Git Hub Copilot or similar
Agile ways of working:
Experience delivering in cross-functional, iterative environments
Note that applications are not being accepted from your jurisdiction for this job currently via this jobsite. Candidate preferences are the decision of the Employer or Recruiting Agent, and are controlled by them alone.
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
Search for further Jobs Here:
×