Databricks Architect

Location:
Ridgefield Park, NJ 07660, onsite position (Monday to Friday at office)

Duration:
Long Term Contract

• Lead the strategic design and architectural roadmap of the Enterprise Data Platform on Databricks/AWS while remaining actively hands-on in design, prototyping, coding, and performance tuning.
• To provide high-level technical consultancy and solution design to internal business stakeholders and external partners, acting as the bridge between complex business requirements and technical implementation.
• To govern the technical quality of the platform by establishing design standards, conducting code reviews, and guiding the engineering team on best practices for Databricks Serverless and AWS integration.
• As part of the role, you will be required to provide technical assurance for all major data initiatives, ensuring that solutions are robust, cost-effective, and compliant with enterprise security standards.
• This role requires 100% onsite presence Monday through Friday and will cover core business hours based on the EST timezone.

• Lead all architecture design and review sessions for strategic Databricks initiatives.
• Actively contribute code to critical architectural components, building complex frameworks, reusable libraries, and core ETL/ELT pipelines using Python and PySpark.
• Lead by example by prototyping Proof of Concepts (PoCs) for new features (e.g., Databricks Asset Bundles, Delta Live Tables) before handing them off for wider team implementation.
• Perform hands‑on performance tuning and debugging of high‑latency jobs by deep‑diving into Spark UI, execution plans, and spill logs.
• Acting as the Technical Design Authority for the Databricks platform, making high‑level decisions regarding the adoption of new Serverless features and AWS services.
• Mentoring and coaching Senior and Junior engineers, fostering a culture of technical excellence and continuous improvement.
• Own the technical relationship with key business stakeholders and translate business strategy into robust data architecture

• Exceptional Customer Facing skills: proven ability to lead workshops, manage stakeholder expectations, and present complex technical solutions to non‑technical audiences.
• Minimum 8‑10 years’ experience in Data Engineering/Architecture
  , with at least 4 years specialized in Databricks and AWS
  .
• Extensive experience designing solutions on the Databricks platform.
• Expert‑level knowledge of Data Lakehouse Architecture, Delta Lake internals, and Spark optimization strategies.
• Proven experience in Data Modeling (Dimensional, Vault, Relational) and designing complex ETL/ELT pipelines.
• Advanced hands‑on coding proficiency in Python and SQL, with the ability to write clean, production‑ready, and testable code.
• Strong experience with Databricks Serverless architecture, Unity Catalog implementation, and security patterns.
• Deep understanding of Cloud Architecture (AWS Well‑Architected Framework), including networking, storage (S3), and IAM.
• Experience in Design Review and Code Review processes, capable of constructive critique and enforcing standards.

• Strong experience working with large‑scale enterprises or as a Consultant for big agencies.
• Certified Databricks Data Architect or AWS Solutions Architect (Professional).
• Excellent organizational and planning skills, with an ability to manage architectural backlogs.
• A best‑in‑class technical leader proactively driving business innovation and process improvements.
• Languages:
  
  Python, SQL, Scala.
• Experience with Enterprise Architecture frameworks (TOGAF).

37.5‑hour week. Monday through Friday,
Onsite
. Work must be performed during EST timezone business hours.