Senior Databricks & AI Architect​/Lead - W2 JK

• Dallas, CA, TX, NJ, NY

Location:

Dallas, TX/NJ/NY/ California (Initial onsite and can converted to Hybrid then after)

Duration:
Long Term contract role

We are seeking a visionary Senior Databricks & AI Architect to lead the evolution of our modern data platform and AI roadmap. You will be responsible for designing scalable Spark-based architectures.

• Architectural Leadership:
  Design and govern end-to-end data architectures on the Databricks Data Intelligence Platform.
• AI Innovation:
  Lead the design of emerging AI workflows, specifically focusing on Agentic AI (multi-agent systems), LLM orchestration (Lang Chain/Llama Index), and RAG patterns tailored for insurance policy and regulatory documents.
• Data Engineering Excellence:
  Standardise data engineering templates (Medallion architecture) and implement robust data modelling (Star/Snowflake, Vault 2.0) to handle structured and unstructured insurance data.
• Governance & Security:
  Ensure all designs comply with industry regulations using Databricks Unity Catalog.

• Experience:
  
  8+ years in Data Engineering/Architecture, with at least 3+ years of hands-on leadership within the Databricks ecosystem.
• Core Tech:
  Expert-level knowledge of the Spark framework (PySpark/Scala), Delta Lake, and Databricks SQL.
• AI/ML Expertise:
  Proven experience building and deploying AI solutions. Deep understanding of the Agentic AI landscape-moving beyond simple chatbots to autonomous, goal-oriented agents.
• Data Management:
  Strong background in data modelling and the creation of reusable ETL/ELT templates and frameworks.