Job Description & How to Apply Below
Key Responsibilities
▪
Assess on-prem Kafka clusters and perform discovery across topics, partitions, consumer groups, throughput patterns, retention policies, and inter-service dependencies to determine migration readiness.
▪
Analyze workloads based on message volume, business criticality, ordering guarantees, latency requirements, and consumer group complexity to determine appropriate MSK cluster placement and configuration.
▪
Plan and provision required AWS MSK infrastructure capacity including broker sizing, storage, networking, and security configurations to support migration, testing, and production execution.
▪
Execute UAT and production parallel testing using Mirror Maker 2 or similar replication tooling, compare message delivery outcomes, capture evidence, and troubleshoot discrepancies through closure.
▪
Perform performance tuning and optimization of MSK clusters to ensure migrated workloads are stable, scalable, efficient, and production ready.
▪
Partner with application engineering, infrastructure, and platform teams to finalize migration plans, consumer/producer cutover approach, validation criteria, and rollback considerations.
▪
Manage release readiness, execute production cutover activities including consumer group migration, offset synchronization, and complete post-release checkout and validation procedures.
▪
Support legacy on-premises Kafka cluster decommissioning after successful migration and validation.
▪
Design, enhance, and implement automation frameworks and AI-assisted solutions to enable repeatable, efficient, large-scale Kafka-to-MSK migrations.
▪
Document migration procedures, operational learnings, risks, and best practices to improve the factory execution model.
Required Qualifications
Basic Qualifications
▪
Education:
Bachelor's or Master's degree in Computer Science, Engineering, Applied Mathematics, or a related quantitative discipline.
▪
Experience:
3–5 years of hands-on engineering experience in a collaborative, team-based environment with distributed messaging systems.
▪
Programming:
Professional proficiency in Python, Java, or a similar programming/scripting language.
▪
Systems:
Strong Unix/Linux fundamentals with the ability to troubleshoot application, deployment, environment, and runtime issues.
▪
Methodology:
Familiarity with SDLC practices, CI/CD delivery models, change management, and Kubernetes-based deployments.
Technical Competencies
Candidates are not expected to be experts in every tool, but should bring strong hands-on experience across several of the following areas:
▪
AWS Services:
Hands-on experience with AWS compute and migration patterns, including ECS, EKS, Lambda, Cloud Watch, and related cloud services.
▪
Apache Kafka:
Deep hands-on experience with Kafka architecture including brokers, topics, partitions, consumer groups, replication, and cluster operations.
▪
AWS MSK:
Practical experience with Amazon Managed Streaming for Apache Kafka including cluster provisioning, configuration, monitoring, and security (IAM, mTLS, SASL/SCRAM).
▪
Migration Tooling:
Experience with Mirror Maker 2, Confluent Replicator, or similar cross-cluster replication tools for data migration and offset synchronization.
▪
Cloud Migration:
Practical understanding of on-prem to cloud migration strategies, workload assessment, testing, cutover, and post-migration validation.
▪
Kafka Ecosystem:
Familiarity with Schema Registry, Kafka Connect, Kafka Streams, and related ecosystem components.
▪
Networking and Security:
Understanding of VPC peering, Private Link, TLS encryption, IAM policies, and network connectivity patterns for hybrid Kafka architectures.
▪
Programming and Automation:
Experience with Python, Java, Shell, or similar languages to build scripts, utilities, and automation frameworks for topic creation, ACL migration, and consumer group management.
▪
APIs and Integration:
Working knowledge of RESTful APIs, API design, and API Gateway patterns.
▪
CI/CD and Version Control:
Strong experience with Git Lab, Jenkins, Maven, or similar tools supporting automated build, test, and deployment pipelines.
▪
Observability:
Familiarity with monitoring, logging, alerting, and troubleshooting using tools such as Cloud…
Note that applications are not being accepted from your jurisdiction for this job currently via this jobsite. Candidate preferences are the decision of the Employer or Recruiting Agent, and are controlled by them alone.
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
Search for further Jobs Here:
×