Senior Databricks Solutions Architect

Prodapt is the largest specialized player in the Connectedness industry. As an AI-first strategic technology partner, Prodapt provides consulting, business reengineering, and managed services for the largest telecom and tech enterprises building networks and digital experiences of tomorrow. A Service Now-invested company, Prodapt has been recognized by Gartner as a Large, Telecom-Native, Regional IT Service Provider. A "Great Place To Work® Certified™" company, Prodapt employs over 5,000 technology and domain experts across the Americas, Europe, India, Africa, & Japan.

Prodapt is part of the 130-year-old business conglomerate The Jhaver Group, which employs over 32,000 people across 80+ locations globally.

This is a onsite role at Dallas, TX, in person interview is must for this role.

We are looking for a Senior Databricks Solution Architect to lead the architecture and technical execution of a Palantir Foundry to Databricks migration
. The candidate will be responsible for assessing the existing Palantir application, defining the target Databricks architecture, mapping existing capabilities to Databricks services, and providing technical leadership throughout the migration.

The solution involves migration of capabilities including data ingestion and transformation, AI-based information extraction from structured and unstructured documents, large-scale data comparison, dashboards and drill-down reporting, chatbot capabilities, and business workflows/applications
.

The ideal candidate should have strong expertise in Databricks, Lakehouse architecture, data engineering, AI/ML workloads, and cloud-native data platforms
, along with the ability to understand and decompose existing Palantir Foundry capabilities into equivalent or improved Databricks-based solutions.

• Define the target-state Databricks architecture for the migrated application.
• Design scalable and secure Lakehouse architecture using Databricks.
• Define the appropriate architecture across Bronze, Silver and Gold layers
  .
• Design data ingestion, transformation, processing and serving patterns.
• Define standards for Delta Lake, Unity Catalog, compute, storage, workloads and environments.
• Design solutions for batch and near-real-time data processing.

• Analyze the existing Palantir Foundry application
  , including its pipelines, datasets, applications, workflows and AI capabilities.
• Create a feature-by-feature mapping between existing Palantir capabilities and Databricks capabilities.
• Identify capabilities that can be migrated directly versus those requiring redesign.
• Define migration patterns for:
  • Data ingestion pipelines
  • Data transformation pipelines
  • Data quality and validation
  • Data comparison and reconciliation
  • AI/LLM-based information extraction
  • Dashboards and reporting
  • Chatbot capabilities
  • Business workflows and actions
  • Document processing
• Develop migration roadmaps, architecture decisions and technical dependencies.
• Identify opportunities to improve scalability, maintainability and performance during migration.

• Design robust data pipelines using Lakeflow / Spark Declarative Pipelines, Apache Spark and Delta Lake
  .
• Design ingestion from databases, APIs, files, documents and other enterprise data sources.
• Establish data quality, validation, lineage and observability mechanisms.
• Design incremental processing and change-data-capture patterns where required.
• Optimize Delta tables, Spark workloads and Databricks compute for performance and cost.

• Architect solutions for extracting information from structured and unstructured documents
  .
• Design RAG and LLM-based solutions where applicable.
• Evaluate appropriate Databricks capabilities for:
  • Doc…