Lead Data Engineer; Snowflake, Airflow and Graph RAG - W2

Overview

Lead Data Engineer (Snowflake, Airflow and Graph RAG) - W2 Role

Location:

New York, NY / Westport, CT (Onsite from Day
1)

Job Type: W2 Contract

Project Duration: 12 months

Note:

Visa Independent candidates are preferred.

• Data Integration and Management:
  Identify and integrate structured, unstructured, and semi-structured data sources. Implement RBAC and develop logical data models, ensuring compliance and data lineage. Document business transformations and establish data quality rules. Create a comprehensive data catalog and register data assets with metadata.
• Design Implementation:
  Translate and implement the architectural blueprint for the Data Factory. Ensure robust data management and quality to support automated investment processes and AI-driven analytics. Implement Neo4j-powered Knowledge Graph and ensure investment decision traceability.
• Advanced Data Solutions:
  Develop and maintain multi-dimensional asset ontology and temporal versioning. Incorporate Graph
  
  RAG for transforming unstructured documents into queryable graph entities.
• Security And Access Control:
  Implement property-based access control and oversee real-time data ingestion pipelines.
• Data Architecture And Management:
  Oversee scalable data warehousing solutions using Snowflake. Develop frameworks for data quality and governance, including metadata registration and sensitivity/security considerations.

• Strong proficiency in Snowflake and Airflow (mandatory).
• Deep knowledge of Graph
  
  RAG and Graph Data Structures.
• Strong background in Data Architecture/Design with experience in market data sources.
• Experience integrating heavy AI-driven architectures. Ability to share knowledge with project and client teams on Cypher, Graph, and RAG traversals.
• AWS experience is a plus.

Note:

The role requires onsite work from Day 1 in New York, NY or Westport, CT. The project duration is 12 months as a contract engagement.