Kafka Architect​/Streaming Data Architect

Job Title

Kafka Architect / Streaming Data Architect

TCS (MEA)

Jeddah, KSA

Full time

• Design enterprise-scale Kafka architecture aligned with Bank Lakehouse (Medallion - Bronze/Silver/Gold).
• Define event-driven architecture patterns for real-time ingestion and processing.
• Establish topic strategy, partitioning, replication, retention policies
  .
• Architect multi-cluster Kafka setup (dev/test/prod, DR strategy).
• Ensure exact point-in-time recovery and fault tolerance
  .

• Lead setup and configuration of:
• Kafka brokers, Zookeeper / Kraft mode
• Kafka Connect framework
• Schema Registry
• Stream processing (Kafka Streams / Spark Streaming / Flink)
• Integrate Kafka with:
• Cloudera Data Platform (CDP)
• Informatica (for ETL/ELT)
• Data Warehouse (Teradata)
• Denodo (for virtualization layer)

• Design pipelines for:
• Event streaming ingestion
• Near real-time batch ingestion
• Build CDC (Change Data Capture) pipelines using Kafka Connect (Debezium).
• Enable ingestion from source systems
  .
• Support incremental and historical loads integration
  .

• Ensure:
• High throughput and low latency
• Support for concurrent consumers without degradation
• Optimize:
• Partition strategy
• Consumer group scaling
• Producer configurations (acks, batching, compression)

• Implement:
• Failover strategies
• Data replication and durability (