Data Engineering Consultant

Position

Description:

We are seeking a skilled Data Engineer to design, build, and maintain scalable, reliable data platforms and pipelines. The ideal candidate will work across cloud‑based and on‑prem data ecosystems, applying strong software engineering principles and modern Dev Ops practices to support analytics, reporting, and advanced data science use cases.

Location - Downtown Toronto (Hybrid - 4 days office) subject to change at any time.

Your future duties and responsibilities:

. Data Pipeline Development:
Design, develop, and maintain robust batch and near‑real‑time data pipelines using Python and SQL to ingest, transform, and serve data at scale.
. ETL & Data Processing:
Implement efficient ETL processes for structured and semi‑structured data using Spark/PySpark and distributed storage systems.
. Data Storage & Access:
Manage and optimize data storage across Mongo

DB, HDFS, Hive, S3, Postgres, and query engines such as Trino.
. Workflow Orchestration:
Build and operate reliable workflows using Airflow, Luigi, and Rundeck with proper scheduling, dependencies, retries, and monitoring.
. Cloud & Platform Engineering:
Develop and deploy data solutions on AWS and enterprise data lake (EDL) platforms, leveraging cloud‑native services and architectures.
. Containerization & Deployment:
Package and deploy data workloads using Docker and orchestrate them with Kubernetes / Open Shift (OCP).
. Data Modeling & Warehousing:
Design data models, schemas, and data warehouses optimized for analytics and reporting performance.
.

Collaboration:

Partner with data scientists, analysts, and application teams to deliver high‑quality, trusted datasets.
. Operational Excellence:
Write clean, testable, and maintainable code; monitor pipelines, troubleshoot failures, and continuously optimize performance and cost.

Required qualifications to be successful in this role:

Core Tools & Technologies
. Programming:
Strong proficiency in Python; working knowledge of R is a plus.
. Data & Storage:
Hands‑on experience with Mongo

DB, S3, Hive, HDFS, Postgres, and distributed processing using Spark / PySpark.
. Query & Analytics:
Advanced SQL skills with modern query engines such as Trino.
. Containers & Platforms:

Experience with Docker, Kubernetes, and Open Shift (OCP).
. Orchestration & Collaboration Tools:
Practical exposure to Airflow, Luigi, Rundeck, Domino Data Labs, and Jupyter Hub.
. Cloud:
Strong experience with AWS and enterprise EDL environments.

Processes & Conceptual Knowledge
. End‑to‑end data pipeline creation and management
. ETL/ELT patterns and best practices
. Data warehousing concepts and performance optimization
. Data modeling (logical & physical)
. Cloud architecture for data platforms
. Solid understanding of software engineering principles (version control, code reviews, testing)
. Experience working in Agile / Dev Ops environments, including CI/CD for data workloads

Nice to Have (Preferred)
. Exposure to streaming or near‑real‑time data processing
.

Experience with data quality frameworks and metadata management
. Familiarity with security, access control, and data governance in cloud platforms
. Prior experience supporting analytics, BI, or ML workloads

Deliverables & Impact
. Scalable, reusable data pipelines and frameworks
. Well‑modeled, high‑quality datasets for analytics and reporting
. Stable, observable, and cost‑optimized data platforms
. Clear documentation and knowledge sharing across teams

** CGI is providing a reasonable estimate of the pay range for this role. The determination of this range includes factors such as skill set level, geographic market, experience and training, and licenses and certifications. Compensation decisions depend on the facts and circumstances of each case. A reasonable estimate of the current range is $95,–$,. This role is an existing vacancy.

#LI-BN

Use of the term ‘engineering’ in this job posting refers to the technical sense related to Information Technology (IT) and does not imply that the individual practices engineering or possesses the requisite license as prescribed by the applicable provincial or territorial engineering regulator. We are seeking individuals with expertise in IT engineering-related functions, but licensure from an engineering regulator is not a prerequisite for this position.

Engineering is a regulated profession in Canada which is restricted in terms of use of titles and designation.

Skills: