Data Warehousing & ETL Engineer

Company Introduction

Company Introduction: We are a financial services company with vast amounts of data from various sources. We currently lack a centralized system for data analysis, which limits our ability to make data-driven decisions and generate meaningful business intelligence.

The Challenge: Our data is siloed and inconsistent, making it nearly impossible to run comprehensive reports and analytics. We need to design and implement a robust data warehouse solution to consolidate our data, ensure its quality, and enable advanced analytics for our business teams. The lack of a proper data strategy is costing us valuable insights.

Objective: The goal is to design, build, and implement a scalable data warehouse, including a reliable ETL process, to centralize our data and provide a foundation for business intelligence.

• Core Tasks:
  • Collaborate with business stakeholders to understand data requirements and design an optimal data warehouse schema.
  • Develop and implement a secure and reliable ETL process to extract data from various sources (e.g., transactional databases, APIs), transform it, and load it into the data warehouse.
  • Ensure data quality and consistency throughout the ETL pipeline.
  • Establish a comprehensive data backup and recovery strategy for the data warehouse.
  • Configure and optimize the data warehouse for high-speed querying and reporting.
  • Provide a detailed support and maintenance plan for the ETL pipeline and data warehouse for the first three months.
  • Document the entire data warehousing architecture, ETL processes, and data lineage.

• Technical Expertise:
  • Proven experience in designing and implementing data warehouse solutions (e.g., using Snowflake, Redshift, Big Query).
  • Expertise in ETL/ELT tools and processes (e.g., Talend, Informatica, custom Python scripts).
  • Strong SQL skills and knowledge of dimensional modeling (Star Schema, Snowflake Schema).
  • Experience with cloud platforms (e.g., AWS, GCP, Azure).
  • Proficiency in scripting languages like Python for data manipulation and automation.

• Excellent analytical and conceptual thinking skills to translate business needs into a technical solution.
• Strong communication skills to work with both technical and non-technical teams.
• Meticulous attention to detail for data quality and consistency.
• A documented data warehouse design and schema.
• A functional ETL pipeline with documentation.
• A comprehensive data quality and backup plan.
• A final report with a roadmap for future data warehousing and business intelligence initiatives.

• Instructions:
  • Submit a proposal outlining your experience with data warehousing and ETL.
  • Provide a portfolio or examples of previous projects.
  • State your hourly rate for this engagement.

• Technical: Data Warehousing, ETL, Snowflake, Redshift, Big Query, SQL, Dimensional Modeling, Python, Data Quality, Backup.
• Soft: Analytical skills, Communication, Problem-solving, Attention to detail.