Data Integration Engineer

Key Qualifications

• Design and implement enterprise-grade data pipelines using Snowflake, including ingestion and transformation
• Expertise in both core and semantic aspects of Snowflake
• Develop complex SQL transformations, stored procedures and dynamic tables to enable near real-time and batch processing
• Implement Snowflake data sharing and data marketplace integrations
• Engineer Snowpipe and Kafka-to-Snowflake streaming ingestion pipelines, handling high-throughput event data at scale
• Optimize Snowflake cluster performance – virtual warehouse sizing, query profiling, clustering keys
• Architectural design, performance tuning, time travel, warehouse concepts – scaling, clustering, micro-partitioning
• Experience with SnowSQL and Snowpipe
• Design and maintain end-to-end ETL/ELT pipelines using Apache Airflow
• Build reusable, parameterized data ingestion pipelines/frameworks
• Perform thorough data quality checks
• Integrate AI/LLMs with data pipelines via Python UDFs or API callouts, enabling text analytics, semantic search and GEN‑AI augmented workflows
• Experience with Python machine learning frameworks (scikit‑learn, PyTorch, Tensor Flow)
• Apply NLP and text‑mining techniques on unstructured data to identify actionable information
• Time‑series forecasting, anomaly detection and propensity modeling
• Data visualization using relevant tools
• Complex SQL query development – joins, self joins, views, materialized views, cursors (also recursive), GROUP BY, PARTITION BY, and SQL performance tuning
• ETL and dimensional data modeling – slowly changing dimensions (SCD Types 1, 2,
  3)
• Strong understanding of schema types, table types (fact/dimension), and design considerations for dimensions versus facts
• Proficiency in Python scripting and programming – Pandas, Numpy, PyParsing, Apache Airflow, and related APIs
• Experience with Tableau Server modules and related data visualization platforms
• Automate data pipelines and maintain high-quality code
• Strong Python programming skills
• Active participation in business discussions to understand requirements, perform impact analysis, and provide suitable solutions

Location:

Sunnyvale, CA

Salary range: $80,000-$140,000 a Year.