Principal Data Engineer; Snowflake & Azure Lakehouse | Technical Lead

Technology Partners is currently seeking a talented

Principal Data Engineer (Snowflake & Azure Lakehouse | Technical Lead).

Do you have experience leading large-scale Snowflake data architectures, building production-grade pipelines, and driving client-facing data engineering engagements?

Let us help you make your next big career move a reality!

You will serve as the hands-on technical lead for a high-stakes, enterprise data modernization program focused on transforming large-scale claims data ingestion into a modern Snowflake + Azure lakehouse architecture. This is a delivery-critical role where you own the technical outcome, lead architecture decisions, build core pipelines, and act as the primary engineering voice with senior client stakeholders. You will operate in a fast-paced, high-visibility environment, balancing execution, leadership, and client engagement.

• Lead the design and implementation ofend-to-end data architecture across Snowflake and Azure environments
• Build and deliver multiple data architecture patterns(Snowflake-native, hybrid Azure + Snowflake, event-driven ingestion, and notebook‑based workflows)
• Implement Medallion Architecture (Raw ? Silver ? Gold) with full lineage, governance, and promotion rules aligned with business definitions
• Design and implementreal‑tim ingestion pipelines using Snowpipe, Event Grid, and secure file ingestion (e.g., PGP-encrypted sources)
• Develop scalable pipelines for high-volume, multi-source data ingestion, transformation, and validation using
  
  Snowpark (Python/Scala)
• Build and optimizedbt Core modelsincluding transformations, testing, and data quality enforcement
• Configure and manage
  
  Azure data services(ADLS Gen2, Azure Data Factory, Blob Storage, Private Endpoints, Managed Identity)
• Develop and maintain operational dashboards for monitoring pipeline health, SLA adherence, and data quality (targeting 99.5% uptime)
• Lead client-facing technical discussions, demos, and working sessions with executive stakeholders
• Translate complex technical decisions into clear business outcomes and data value narratives
• Scope work, estimate delivery timelines, and manage technical risks and change requests
• Lead and mentor supporting engineering resources while ensuring high-quality deliverables
• Drive AI/ML integration initiatives including
  
  Snowflake Cortex, Streamlit, and Azure AI services
• Ensure all architecture, pipelines, and transformations are well-documented and production-ready

• 5+ years of experience indata engineering, with at least 2+ years in atechnical lead or principal role
• Deep, hands‑on expertise in:
  • Snowflake architecture and optimization
  • Snowpark (Python/Scala) for pipeline development
  • Advanced SQL(complex transformations, performance tuning, schema design)
  • dbt Core(modeling, testing, incremental processing, source management)
• Strong experience with Azure cloud ecosystem, including:
  • ADLS Gen2, Azure Data Factory, Azure Functions
  • Event Grid, Blob Storage, Private Endpoints, Managed Identity
• Proven experience building event-driven and streaming data pipelines(e.g., Snowpipe)
• Experience implementing
  
  Medallion/Lakehouse architecture patterns in production environments
• Experience with data governance tools(e.g., Microsoft Purview or equivalent)
• Proven ability to operate inclient-facing consulting environments, managing stakeholders and delivering under pressure
• Strong leadership, communication, and problem‑solving skills
• Ability to balance speed of delivery with system reliability and scalability

• Experience with AI/ML integration on data platforms(Snowflake Cortex, Streamlit, Azure AI Foundry, or similar)
• Exposure toPGP/GPG encryption workflows in automated pipelines
• Familiarity with…