Senior Data Engineer

Responsibilities

• Implement methods for automation of all parts of the pipeline to minimize labor in development and production.
• Analyze complex data, organizing raw data, and integrating massive datasets from multiple data sources to build analytical domains and reusable data products.
• Work with architects to evaluate and productionalize data pipelines for data ingestion, curation, and consumption.
• Communicate with stakeholders to formulate business problems as technical data requirements, identify and implement technical solutions while ensuring key business drivers are captured in collaboration with product management.
• Develop exceptional analytical data products using both streaming and batch ingestion patterns on Google Cloud Platform with solid data warehouse principles.
• Serve as the Subject Matter Expert in Data Engineering with a focus on GCP native services and other well integrated third-party technologies.

• Master’s degree or foreign equivalent in Computer Science or related field and 5 years of experience in the job offered or related occupation.
• 5 years of experience with SQL Development for analyzing complex data, organizing raw data, and integrating massive datasets from multiple data sources into data factory data lake.
• 5 years of experience with Analytics and data product development to formulate business problems as technical data requirements, identify and implement technical solutions while ensuring key business drivers are captured in collaboration with product deliverables.
• 3 years of experience with Google Cloud Platform (GCP) experience with solutions designed and implemented at production scale.
• 2 years of experience with GCP native (or equivalent) services including Big Query, Google Cloud Storage, Pub Sub, Dataflow, Dataproc, and Cloud Build to evaluate and productionalize data pipelines for data ingestion, curation, and consumption.
• 2 years of experience working with Airflow for scheduling and orchestration of data pipelines.
• 2 years of experience utilizing Terraform to provision Infrastructure as Code.
• 2 years of experience using Java or Python for development of cloud run http and event-driven functions which handle http requests and events from cloud environment and use event triggers.

• SQL Development
• data analysis
• data integration
• data product development
• data ingestion
• data curation
• data consumption
• cloud development
• Infrastructure as Code
• event-driven programming

• communication
• collaboration
• problem-solving
• stakeholder engagement
• analytical thinking

• Master’s degree in Computer Science