Senior GCP Data Engineer

Overview

We're looking for a highly skilled Data Analyst to join our team, focusing on the migration of our data platform from Teradata to Google Cloud's Big Query. The ideal candidate will have deep expertise in both Teradata and Big Query, with a proven track record of analyzing, understanding, and translating complex business logic from legacy systems to modern cloud environments. This role requires a blend of technical prowess, business acumen, and problem-solving skills to ensure a smooth and accurate transition.

• Teradata Script Analysis:
  Analyze and interpret existing Teradata scripts, including BTEQ, SQL, and stored procedures, to understand the underlying business logic and data transformation processes.
• Business Logic Deduction:
  Collaborate with business stakeholders to deduce and document the business rules, data lineage, and purpose of existing Teradata scripts and data models.
• Teradata Script Modification:
  Modify and optimize Teradata scripts as needed to meet current and future business requirements.
• Big Query Conversion:
  Translate complex Teradata SQL and stored procedures into Google Standard SQL for execution within Big Query.
• ETL/ELT Pipeline Development:
  Design, develop, and implement efficient data loading and transformation processes to migrate data from on-premise sources to Big Query.
• Data Validation & Quality:
  Perform thorough data validation and quality checks to ensure data integrity and accuracy post-migration.
• Documentation:
  Create comprehensive documentation for all migrated scripts, data models, and business logic.

• Experience:
  
  Minimum of 7+ years of experience as a Data Engineering/data analyst or a related role.
• Teradata Expertise:
  Strong proficiency in Teradata SQL, BTEQ scripts, stored procedures, and the Teradata architecture.
• Big Query Proficiency:
  Solid understanding and practical experience with Google Cloud Platform (GCP) and specifically with Big Query. Familiarity with bq command-line tool, Big Query's data loading mechanisms, and data partitioning/clustering is essential.
• SQL Mastery:
  Exceptional SQL skills with the ability to write complex, performant queries and understand query execution plans in both Teradata and Big Query.
• Analytical
  
  Skills:
  
  Excellent problem-solving and analytical skills with the ability to reverse-engineer complex logic and understand data relationships.
• Communication:
  Strong verbal and written communication skills to effectively interact with technical and non-technical stakeholders.

• Experience with other GCP services such as Cloud Storage, Dataflow, and Dataproc.
• Knowledge of scripting languages like Python or Shell for automation purposes.
• Familiarity with data warehousing concepts and data modeling techniques.