Lead Architect

Lead Architect – Azure & AI

Fractal is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. Fractal is building a world where individual choices, freedom, and diversity are the greatest assets; an ecosystem where human imagination is at the heart of every decision. Where no possibility is written off, only challenged to get better. We believe that a true Fractalite is the one who empowers imagination with intelligence.

We are looking for a Lead Architect – Azure & AI to join our growing data engineering practice. In this role, you will architect and deliver large-scale, cloud-native data solutions on Azure and Snowflake, enabling advanced analytics, ML/AI workloads, and enterprise data governance. You will work closely with cross-functional stakeholders to design resilient, scalable data platforms that drive business impact for Fortune 500 clients.

Key Responsibilities

• Design, build, and optimize end-to-end data pipelines leveraging Azure Data Factory, Databricks, and Snowflake for large-scale data ingestion, transformation, and loading.
• Develop and maintain ETL/ELT workflows for structured, semi-structured (JSON, XML, Parquet), and unstructured data from on-premises systems, cloud platforms, APIs, flat files, RDBMS, and streaming sources.
• Write efficient, production-grade code in Python (PySpark) and SQL for advanced data transformations, workflow orchestration, and automation.
• Architect scalable Snowflake solutions including schema design, Virtual Warehouse configuration, query performance tuning, and cost-efficient data storage and retrieval strategies.
• Leverage Azure Synapse and Snowflake for advanced analytics and data warehouse solutions, including MPP (Massively Parallel Processing) architectures.
• Partner with business and technology stakeholders to design Data Mart and Data Lake solutions supporting analytics, reporting, and ML/AI use cases.
• Implement data quality frameworks, governance policies, lineage tracking, and metadata management across multiple environments.
• Optimize Spark workloads on Databricks for distributed processing at scale; deploy and manage reusable Notebooks and Jobs in production.
• Integrate No
  
  SQL stores (Cosmos DB, MongoDB, etc.) and build hybrid solutions spanning Snowflake and Azure services.
• Leverage Azure Cognitive Search and Elasticsearch for indexing and unstructured data search integration.
• Drive Dev Ops best practices including CI/CD pipelines, source control (Git), and automated deployments via Azure Dev Ops and Jenkins.
• Ensure adherence to enterprise data security, cataloging, and access-control policies across all data assets.

Required Skills

• 8+ years of experience as a Data Engineer with deep expertise in Azure Data Services (ADF, Synapse, Databricks) and Snowflake.
• Strong background in the Insurance domain with hands-on Azure Data Engineering experience (ADB + PySpark/Python + Snowflake).
• Proficiency in SQL (complex queries, stored procedures, functions, views) and Python/PySpark programming.
• Proven experience designing and implementing ETL/ELT pipelines with Azure Data Factory and orchestrating complex, multi-step workflows.
• Comprehensive Snowflake expertise:
  Star/Snowflake schema design, role-based security, performance tuning, query optimization, and pipeline features (Snowpipe, Tasks, Streams).
• Solid understanding of Spark architecture with hands-on experience in performance tuning, data partitioning, and distributed processing.
• Experience integrating data from RDBMS, SFTP, APIs, Kafka, Event Hub, and big-data platforms (Hadoop, Hive).
• Familiarity with No
  
  SQL database design (Cosmos DB or equivalent) and streaming data pipeline patterns.
• Knowledge of ML/AI data preparation workflows; ability to refactor traditional ML code for distributed execution on Spark/Databricks.
• Strong command of Agile methodologies, Dev Ops practices, and CI/CD deployment frameworks.

Preferred Skills

• Exposure to data search and indexing technologies such as Elasticsearch or Azure Cognitive Search.
• Azure Data Engineer Associate certification (DP-200/201/203) or Snow Pro Core/Advanced Architect certification.
• Experience working on data…