Senior Data Engineer - GCP

Select how often (in days) to receive an alert:

Requisition
Join a purpose driven winning team, committed to results, in an inclusive and high-performing culture.

The Wealth Data engineering team within the Global Wealth Engineering (GWE) is the key team in meeting the operational data needs of the various stakeholders within Wealth Management. The Lead Data Engineer will play a key role in designing and implementing data solutions using Big Data, Hadoop (Cloudera) and Google Cloud, working closely with the enterprise data team, data architects, solution architects, business systems analysts and other data engineers.

• Help development efforts in ingesting and transforming data from various sources. Work in the Big Data/Hadoop environment, writing code, building scripts, drafting specifications and responsible for end‑to‑end delivery of data to the Enterprise Data Lake.
• Build distributed, reliable and scalable data pipelines to ingest and process data from multiple data sources.
• Design and build the data platform using Google Cloud Platform (GCP) services such as Data Proc
  , Dataflow
  , CloudSQL
  , Big Query
  , Cloud Spanner
  , combined with third‑party tools such as Spark, Apache Beam/Composer,
  DBT
  , Cloud Pub/Sub,
  Confluent Kafka
  , Cloud Storage, Cloud Functions, Git Hub.
• Design and implement data ingestion patterns that support batch, streaming and API interfaces on both ingress and egress.
• Guide a team of data engineers and develop frameworks and custom code following best practices to meet demanding performance requirements.
• Take a lead in designing and building production data pipelines from ingestion to consumption using GCP services, Java, Python, Scala, Big Query, DBT, SQL, etc.
• Experience using Cloud Dataflow in Java/Python for deploying streaming jobs in GCP as well as batch jobs using text/JSON files and writing them to Big Query
  .
• Build and manage data pipelines with a deep understanding of workflow orchestration, task scheduling and dependency management.
• Provide end‑to‑end technical guidance and expertise on effectively using Google Cloud to build solutions, creatively applying cloud infrastructure and platform services to solve business problems, and communicate these approaches to various business users.
• Guide application logging, notification, job monitoring, and performance monitoring implementation.

• 2+ years of data engineering experience, performance optimization for large OLTP applications.
• Knowledge of Hadoop HDFS, Hive, Pig, Flume, and Sqoop.
• Experience with GCP managed services such as Data Proc, Dataflow, Big Query/DBT, Cloud Spanner, Cloud SQL, Cloud Pub/Sub.
• F with GCP databases (SQL, Spanner, Postgre
  
  SQL).
• Working knowledge of HQL.
• Understanding of Hadoop concepts.
• Experience with relational/No
  
  SQL databases.
• Knowledge of streaming and technologies such as Kafka, Spark‑streaming.
• Experience with Infrastructure as Code practices and frameworks like Terraform.
• Knowledge of Java microservices and Spring Boot.
• Strong architecture knowledge and ability to provide technical solutions for cloud infrastructure.
• Experience developing and scaling Java REST services with Spring.
• Good communication and problem‑solving skills; ability to convey ideas to business and technical teams.

• Understanding of Wealth business line and the various data domains required for building end‑to‑end solutions.

• Diversity, Equity, Inclusion & Allyship – we strive to create an inclusive culture where every employee is empowered to reach their fullest potential, respected for who they are, and embraced through bias‑free practices and inclusive values across Scotiabank.
• Accessibility and Workplace Accommodations – we value the unique skills and experiences each individual brings, and are committed to creating and maintaining an inclusive and accessible environment for everyone.
• Upskilling through online courses, cross‑functional development opportunities, and tuition assistance.
• Competitive Rewards program including bonus, flexible vacation, personal, sick days and benefits starting on day one.
• Dynamic Ecosystem – free tea & coffee, universal washrooms, and plenty of space for team collaboration.

Location(s):
Canada :
Ontario :
Toronto