Kafka Architect​/Streaming Data Architect

Job Title – Kafka Architect / Streaming Data Architect

Company – TCS (MEA)

Job type – Full time

Tata Consultancy Services (TCS) is an IT services, consulting and business solutions organization that has been partnering with many of the world’s largest businesses in their transformation journeys for over 50 years. TCS offers a consulting-led, cognitive powered, integrated portfolio of business, technology and engineering services and solutions. This is delivered through its unique Location Independent Agile™ delivery model, recognized as a benchmark of excellence in software development.

A part of the Tata group, India's largest multinational business group, TCS has over 616,171 of the world’s best-trained consultants with 157 nationalities in 53 countries.

• Design enterprise-scale Kafka architecture aligned with Bank Lakehouse (Medallion - Bronze/Silver/Gold).
• Define event-driven architecture patterns for real-time ingestion and processing.
• Establish topic strategy, partitioning, replication, retention policies.
• Architect multi-cluster Kafka setup (dev/test/prod, DR strategy).
• Ensure exact point-in-time recovery and fault tolerance.

• Lead setup and configuration of:
• Kafka Connect framework
• Schema Registry
• Integrate Kafka with:
• Cloudera Data Platform (CDP)
• Data Warehouse (Teradata)
• Enable ingestion from source systems.
• Support incremental and historical loads integration.

• Ensure:
• High throughput and low latency
• Support for concurrent consumers without degradation
• Optimize:
• Consumer group scaling
• Producer configurations (acks, batching, compression)

• Implement:
• Failover strategies
• Data replication and durability (99.9% uptime requirement)
• Handle:
• Hardware failures
• Design retry, dead-letter queues (DLQ), error handling frameworks

• Enforce:
• Role-Based Access Control (RBAC)
• Encryption (in transit & at rest)
• Data masking/tokenization support (aligned with classification)
• Integrate with:
• Metadata and lineage tools
• Governance frameworks (NDMO/NDI compliance)

• Implement monitoring using:
• Cloudera Manager
• Kafka metrics (JMX, Prometheus, Grafana)
• Define:
• Alerting thresholds
• Ensure observability for:
• Throughput and error rates

• Assess existing Kafka pipelines and components.
• Identify:
• Redesign/rebuild where required.
• Ensure alignment with target architecture and data models (FSLDM/FSAS).

• Ensure:
• Real-time data availability for downstream reporting.
• Support:
• Query performance optimization
• Data consistency across Bronze/Silver/Gold

• Deliver:
• Topic catalog and schema definitions
• Maintain:
• Source-to-target mappings
• Architecture governance boards

• Cloudera Kafka (mandatory)
• Kafka Connect, Kafka Streams
• Schema Registry (Avro/Protobuf/JSON)

• Cloudera (CDP)
• Apache Flink (preferred)

• REST APIs, Microservices integration
• ETL/ELT Design patterns

• Kubernetes / Docker (optional but preferred)

• Experience in BFSI domain
• Understanding of regulatory reporting

• Strong stakeholder management (business + IT)
• Ability to work in multi-vendor environment
• Experience in large-scale transformation programs
• Strong problem-solving and RCA skills

Final date to receive applications: 30
- July
-2026