Principal Data Solutions Architect- AI​/ML

We are seeking an experienced Data & Analytics Solutions Architect to lead the design and implementation of modern enterprise data platforms, advanced analytics solutions, and AI-driven architectures. The ideal candidate will have extensive experience in architecting large-scale open-source data ecosystems, distributed data processing platforms, Data Lakehouse architectures, and AI/ML-enabled analytics solutions.

• Engage with business and technology stakeholders to understand data strategy, platform requirements, architecture goals, and transformation priorities.
• Design and define end-to-end architecture for enterprise open-source data platforms supporting batch, streaming, analytics, and AI workloads.
• Lead modernization initiatives covering data lakes, lakehouse platforms, distributed processing frameworks, and enterprise data ecosystems.
• Architect scalable, secure, highly available, and resilient data platforms using open-source technologies and industry best practices.
• Design data ingestion, integration, streaming, storage, and consumption patterns using technologies such as Spark, Kafka, Flink, NiFi, and Trino.
• Establish data governance, security, metadata, lineage, privacy, and observability standards across the platform landscape.
• Evaluate and recommend open-source technologies based on business requirements, scalability, supportability, interoperability, and total cost of ownership.
• Define architecture standards, reference architectures, reusable frameworks, CI/CD practices, and Data Ops operating models.
• Provide technical leadership across data modeling, data engineering, platform implementation, migration, testing, deployment, and production readiness activities.
• Drive platform performance optimization, capacity planning, reliability engineering, and operational excellence initiatives.
• Conduct architecture reviews, identify risks early, and guide delivery teams on mitigation strategies and best practices.
• Support solutioning, RFP responses, effort estimation, technical presentations, and client workshops for strategic opportunities.
• Collaborate with engineering teams, platform vendors, open-source communities, and client stakeholders to deliver scalable and sustainable solutions.
• Mentor architects and engineers, contribute to technical capability development, and establish reusable accelerators, playbooks, and best practices.
• Support hiring, technical assessments, and knowledge-sharing initiatives across Data Engineering, Open-Source Platforms, and Modern Data Architecture.
• Assess candidate use cases with business, technology, data, and risk stakeholders; recommend BI, rules, analytics, ML, or GenAI based on feasibility, risk, cost, and measurable business outcomes.
• Integrate data, ML, and GenAI reference architectures covering data sources, feature pipelines, models or LLMs, vector stores, serving layers, consuming applications, and enterprise integration points.
• Partner with platform and operations teams to product ionize AI ML and LLM workloads on Kubernetes.

• 14+ years of experience in Enterprise Data Architecture, Open-Source Data Platforms, Distributed Data Processing, and Data Engineering.
• Proven experience in Enterprise Solution architecture using Open source technology stack across the Data to Insights end to end value chain.
• Strong hands-on expertise in Apache Spark, PySpark, Kafka, Flink, Airflow, NiFi, Trino, SQL, Python, Scala, and Java.
• Proven experience in designing and implementing open-source data platform architectures, including component selection, integration, scalability, interoperability, and platform modernization.
• Deep understanding of batch, streaming, and real-time data processing architectures, including performance tuning, workload optimization, and failure recovery.
• Experience building modern Data Lakehouse architectures using Apache Iceberg, Hudi, Delta Lake, object storage, and open table formats.
• Strong knowledge of event-driven architectures, CDC, APIs, schema management, and enterprise integration patterns.
• Expertise in Data Ops practices, including Airflow orchestration, CI/CD, automated testing, observability, metadata management, lineage, and platform operations.
• Experience with PostgreSQL, Cassandra, MongoDB, Open Search, and enterprise analytics platforms, including data modeling, indexing, workload…