Data Engineer Consultant
Job in
Jeddah, Saudi Arabia
Listed on 2026-06-07
Listing for:
Total-TECH Co
Full Time
position Listed on 2026-06-07
Job specializations:
-
IT/Tech
Data Engineering, Cloud Computing: Infrastructure & Operations
Job Description & How to Apply Below
Enterprise Data Ingestion and Data Engineering
- Design and build scalable reusable ingestion pipelines realtime and batch on GCP GCS Big Query Dataflow; develop parameterized pipelines using Google‑native services and or Informatica IDMC mappings and taskflows.
- Implement CDC patterns, idempotent loads, late‑arriving data handling and schema evolution; optimize Big Query ingestion strategies between batch and streaming, partitioning, and clustering.
- Establish version control, CI/CD, and environment promotion for dev, test, and prod.
- GCP‑native pipelines (Dataflow, Composer, Cloud Run) and IDMC mappings/taskflows with Big Query schema designs across raw, stage, and curated zones; implement partitioning, clustering, CI/CD pipelines, deployment artifacts, and operational runbooks.
- Profile source data to analyze data quality and patterns.
- Design field‑level mappings, business rules, joins, aggregations, and derivations.
- Implement transformations using Big Query SQL, Dataflow, or IDMC transformation logic.
- Source‑to‑Target Mapping documentation and transformation mappings (STM).
- Build and parameterize end‑to‑end workflows using Cloud Composer, Airflow, or IDMC taskflows.
- Define job dependencies, schedules, SLAs, and failure‑handling strategies.
- Implement retries, backoff strategies, checkpoints, and restartability.
- Integrate monitoring, logging, and alerting with Cloud Monitoring, Cloud Logging, and Chat Ops tools.
- Production‑ready orchestration workflows with environment‑aware parameters; scheduling calendars, dependency diagrams, and SLA matrices.
- Alerting, monitoring dashboards, and operational SOPs.
- Monitor pipeline health, data freshness, volumes, and anomaly patterns; support and monitor data pipelines during off‑hours and weekends.
- Track and manage SLAs related to runtime failures, cost, and data latency.
- Optimize Big Query performance through query refactoring, partition pruning, materialized views.
- Manage and optimize GCP costs: storage lifecycle, slot usage, query optimization, caching.
- Daily and weekly operational and SLA reports, performance and cost optimization plans with before‑and‑after metrics, Big Query tuning guidelines, and data materialization strategy.
- 5 years of experience as a Data Engineer or ETL Developer.
- Strong experience with Google Cloud Platform, Big Query, GCS, Data Stream, Dataflow, Cloud Composer, IAM, Cloud Monitoring.
- Proficiency in SQL, Big Query‑optimized and data modeling for analytics.
- Hands‑on experience building production‑grade scalable data pipelines.
- Experience with CI/CD, Git‑based version control, and Dev Ops practices.
- Optional:
Experience with Informatica Intelligent Data Management Cloud, IDMC, CDI, Mass Ingestion, Taskflows. - Bachelor's degree in Computer Science, Information Technology, Information Systems, or a related field.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×