Generative AI Consultant

Position Overview

This role is for a Senior Databricks Data Architect (AWS) in Ridgefield Park, NJ, for a 12‑month contract at $X/hour. Requires 8‑10 years of experience in Data Engineering/Architecture, expertise in Databricks, AWS, Python, and strong customer‑facing skills.

Ridgefield Park, New Jersey 07660; 12‑month onsite contract; 100% onsite presence Monday‑through‑Friday; aligned with EST timezone.

Lead the strategic design and architectural roadmap of the Enterprise Data Platform on Databricks/AWS while remaining actively hands‑on in design, prototyping, coding, and performance tuning. Provide high‑level technical consultancy and solution design to internal business stakeholders and external partners, acting as the bridge between complex business requirements and technical implementation. Govern the technical quality of the platform by establishing design standards, conducting code reviews, and guiding the engineering team on best practices for Databricks Serverless and AWS integration.

Provide technical assurance for all major data initiatives, ensuring that solutions are robust, cost‑effective, and compliant with enterprise security standards.

• Lead all architecture design and review sessions for strategic Databricks initiatives.
• Actively contribute code to critical architectural components, building complex frameworks, reusable libraries, and core ETL/ELT pipelines using Python and PySpark.
• Lead by example by prototyping Proof of Concepts (PoCs) for new features (e.g., Databricks Asset Bundles, Delta Live Tables) before handing them off for wider team implementation.
• Perform hands‑on performance tuning and debugging of high‑latency jobs by deep‑diving into Spark UI, execution plans, and spill logs.
• Define and enforce coding standards, data modeling best practices (Medallion Architecture), and CI/CD protocols for the development team.
• Conduct deep‑dive technical reviews of designs and code produced by the engineering team to ensure efficiency, maintainability, and security.
• Design the strategy for Unity Catalog governance, ensuring granular data access controls and compliance with data privacy regulations.
• Act as the Technical Design Authority for the Databricks platform, making high‑level decisions regarding the adoption of new Serverless features and AWS services.
• Mentor and coach Senior and Junior engineers, fostering a culture of technical excellence and continuous improvement.
• Own the technical relationship with key business stakeholders and translate business strategy into robust data architecture.
• Engage and collaborate with Executive Stakeholders to present technical roadmaps and demonstrate the business value of data initiatives.
• Seek opportunities to develop knowledge in emerging technologies (e.g., Generative AI on Databricks, Vector Databases) and evaluate their impact on the business.
• Provide guidance to the Change Management board regarding the risk and technical impact of proposed architectural changes.
• Develop strategies to architect resilience, high availability, and disaster recovery into the data platform.
• Develop clear communication channels with C‑level executives and Product Managers to ensure technical strategies align with business goals.

• Head of Data & Analytics / CIO
• Business Unit Directors & Product Managers
• Lead Data Engineers and Data Scientists
• Enterprise Security & Infrastructure Architects

Ownership of the Technical Roadmap and Architecture for the Data Estate; maintain excellent relationships with all key liaisons; responsible for architectural decisions impacting critical data assets; primary technical advisor in the CST timezone.

Degree level in Computer Science, Engineering, or equivalent (Master's degree preferred).

• Exceptional Customer‑Facing skills: proven ability to lead workshops, manage stakeholder expectations, and present complex technical solutions to non‑technical audiences.
• Minimum 8‑10 years of experience in Data Engineering/Architecture, with at least 4 years specialized in Databricks and AWS.
• Expert‑level knowledge of Data Lakehouse…