GCP Data Engineer

: up to 12 months contract with possible extension

Pay rate: $50 to $65/hr the pay rate may differ depending on your skills, education, experience, and other qualifications.

Featured Benefits: Medical Insurance in compliance with the ACA. 401(k). Sick leave in compliance with applicable state, federal, and local laws.

We are looking for an onsite, hands-on Data Engineer with strong GCP and SQL skills who can start as soon as possible.

Job Description:

• Design, develop, and maintain scalable data pipelines using Google Cloud Platform services.
• Build and manage ETL/ELT pipelines using Cloud Dataflow, Dataproc, Cloud Composer, and Big Query.
• Develop complex SQL queries, stored procedures, views, and data transformations in Big Query.
• Implement batch and real-time data processing using Apache Spark, PySpark, and Dataflow.
• Develop data ingestion pipelines from multiple sources such as APIs, databases, files, and streaming systems.
• Work with Cloud Storage (GCS) for data lake implementation and manage data in structured, semi-structured, and unstructured formats.
• Design and optimize Big Query data models, partitioning, clustering, and performance strategies.
• Implement real-time streaming pipelines using Pub/Sub and Dataflow.
• Orchestrate and schedule data workflows using Cloud Composer (Apache Airflow).
• Develop PySpark jobs on Dataproc for large-scale distributed data processing.
• Implement data quality, validation, reconciliation, and error-handling frameworks.
• Optimize data pipelines for performance, scalability, reliability, and cost efficiency.
• Implement CI/CD pipelines for data engineering workloads using Cloud Build, Git, and Terraform.
• Manage security and access using IAM, service accounts, Secret Manager, and encryption.
• Monitor pipelines and troubleshoot failures using Cloud Monitoring, Cloud Logging, and alerting.
• Collaborate with data architects, analysts, data scientists, and business teams to understand data requirements.
• Implement data governance, metadata management, lineage, and security best practices.
• Migrate on-premises or other cloud data platforms to GCP, including data warehouses and ETL workloads.
• Automate infrastructure and GCP resources using Terraform / Infrastructure as Code.
• Create technical documentation for data pipelines, data models, architecture, and operational procedures.