Senior Data Architect

Assistant Manager @ EXL | Shaping Tomorrow’s IT Landscape Through Insight, Innovation & Delivery

We are seeking a Senior Data Architect with deep Big Data Engineering expertise to design and modernize large-scale, cloud-native data platforms
. This role emphasizes distributed data processing, real-time pipelines, data platform automation
, and GenAI enablement on top of strong Big Data foundations.

• Architect and govern enterprise Big Data platforms (data lake, lakehouse, warehouse, real-time).
• Design high-volume, high-velocity data pipelines using batch and streaming frameworks.
• Lead implementation of distributed processing architectures (Spark, PySpark, EMR).
• Build event-driven and real-time streaming solutions (Kafka, Kinesis, Flink).
• Define ETL/ELT patterns
  , metadata-driven pipelines, and reusable ingestion frameworks.
• Optimize performance, scalability, fault tolerance, and cost across Big Data workloads.
• Integrate GenAI architectures (LLMs, embeddings, vector databases, RAG) with enterprise data lakes.
• Ensure security, governance, lineage, and compliance across data platforms.
• Provide hands‑on leadership and technical mentoring to data engineering teams.

• 12+ years in Big Data Engineering / Data Architecture roles.
• Expert-level experience with Spark, PySpark, SQL
  , and distributed compute engines.
• Strong knowledge of AWS Big Data stack
  : S3, EMR, Glue, Athena, Redshift, Lambda, Step Functions.
• Hands‑on experience with Snowflake (performance tuning, data sharing, optimization).
• Expertise in streaming platforms
  :
  Kafka, Kinesis, Flink, or Spark Streaming.
• Strong experience with data modeling (dimensional, Data Vault 2.0).
• Proficiency in Python
  , schema evolution, partitioning, and data versioning.
• Experience with orchestration and automation tools (Airflow, Dagster, CI/CD).
• Working knowledge of GenAI data integration (feature stores, vector DBs, RAG pipelines).

Experience with
Agile delivery and leading globally distributed engineering teams.

Mid-Senior level

Full-time

Information Technology

Information Services