Principal Architect

Principal Architect What you will do

Let’s do this. Let’s change the world. In this vital role you will play a pivotal role in building and scaling our machine learning models from development to production. Your expertise in both machine learning and operations will be essential in creating efficient and reliable ML pipelines. A background in data engineering, including experience with data pipelines and distributed data processing, is a strong plus.

• Lead the end-to-end design, development, and delivery of machine learning and Generative AI (GenAI) solutions, leveraging Databricks, Apache Spark, SQL, and Python for scalable data processing, feature engineering, and model development from problem framing to production deployment and business impact realization.
• Act as an Architect for large-scale Data Engineering and ML/GenAI initiatives
  , driving architecture decisions across lakehouse platforms (Databricks), distributed compute (Spark), and cloud ecosystems (AWS/GCP/Azure) to ensure scalability, reliability, and long‑term maintainability.
• Design and implement advanced data pipelines and AI systems
  , including batch and streaming data processing (Spark), data modeling (SQL), and ML workflows (Python), along with multi‑agent architectures, reasoning workflows, tool integration, and autonomous decision‑making systems.
• Build and optimize robust data foundations for AI by developing high-quality, scalable ETL/ELT pipelines in Databricks
  , ensuring data availability, consistency, and performance for downstream ML/GenAI use cases.
• Define and institutionalize evaluation, validation, and governance frameworks for ML/GenAI systems, including model performance tracking, prompt evaluation, safety guardrails, hallucination mitigation, and compliance.
• Partner directly with business stakeholders and product leaders to translate objectives into data-driven AI/ML solutions
  , ensuring measurable value through well‑defined data pipelines, KPIs, and experimentation frameworks.
• Establish and enforce best practices in MLOps, LLMOps, Data Ops, and Dev Ops
  , including CI/CD pipelines,
  Databricks workflows
  , monitoring, observability, reproducibility, and cost optimization.
• Architect and oversee scalable cloud‑based data and AI platforms
  , integrating Databricks Lakehouse, Spark processing layers, and cloud‑native services for unified analytics and AI workloads.
• Drive experimentation strategy, including A/B testing, prompt optimization, and data‑driven iteration
  , leveraging SQL analytics and Python‑based experimentation frameworks
  .
• Provide mentorship to L4 and L5 engineers in data engineering (Spark, SQL, Databricks) and AI/ML development (Python, GenAI frameworks), including design reviews, code reviews, and career guidance.
• Lead cross‑functional collaboration across data engineering, data science, platform engineering, and business teams to deliver integrated, production‑grade AI solutions.
• Stay at the forefront of advancements in data engineering (Spark ecosystem, lakehouse architectures) and Generative AI/agentic systems
  , driving adoption of new technologies and best practices.

• Doctorate degree and 2 years of experience OR
• Master’s degree and 4 years of experience OR
• Bachelor’s degree and 6 years of experience OR
• Associate’s degree and 10 years of experience OR
• High school diploma / GED and 12 years of experience

• Deep expertise in machine learning, deep learning, and Generative AI (LLMs, transformers, embeddings, fine‑tuning techniques).
• Proven track record of leading and delivering production‑grade ML/GenAI systems end‑to‑end with measurable business impact with strong experience in designing scalable system architectures for ML and GenAI, including distributed systems and high‑throughput pipelines.
• Expertise in MLOps/LLMOps ecosystems (MLflow, Kubeflow, Airflow, CI/CD, Docker, Kubernetes).
• Strong system design, architecture, and problem‑solving skills with the ability to operate independently and lead large initiatives.
• Demonstrated proficiency in leveraging cloud platforms (AWS, Azure, GCP) for data engineering solutions. Strong understanding of cloud architecture…