Kafka Admin

Kafka Admin

Experienced Big Data Administrator responsible for managing, supporting, automating, and optimizing enterprise-scale Hadoop, Kafka, AWS cloud, and Dev Ops platforms. The role focuses on ensuring high availability, performance, security, scalability, and operational excellence across big data ecosystems while collaborating with development, architecture, and business teams.

Key Responsibilities:

• Kafka Administration
• Deploy, configure, and manage Apache Kafka clusters and AWS MSK environments.
• Monitor broker health, partitions, replication factors, and consumer lag.
• Perform capacity planning and cluster scaling activities.
• Manage Kafka security using SSL, SASL, ACLs, and encryption standards.
• Troubleshoot producer, consumer, and broker performance issues.
• Support Kafka Connect, Schema Registry, Cruise Control, and Mirror Maker implementations.
• AWS Cloud Administration
• Manage cloud infrastructure services including EC2, S3, IAM, VPC, EBS, Cloud Watch, Cloud Trail and AWS Glue.
• Support AWS Managed Streaming for Kafka (MSK), EMR, Lambda, and Airflow environments.
• Implement cloud security best practices and governance controls.
• Perform infrastructure provisioning and automation using Infrastructure as Code (IaC).
• Monitor cloud resource utilization and optimize operational costs.
• Dev Ops & Automation
• Design and maintain CI/CD pipelines using Jenkins, Git Hub Actions, Git Lab CI, or similar tools.
• Automate infrastructure deployment using Terraform, Cloud Formation, and Ansible.
• Manage source control repositories and release processes.
• Implement monitoring and alerting solutions using Prometheus, Grafana, Splunk, ELK, or Cloud Watch.
• Support containerization technologies such as Docker and Kubernetes.
• Develop automation scripts using Python, Shell, or Bash.
• Operations & Support
• Provide Level 2 and Level 3 production support.
• Participate in on-call support rotations and incident management activities.
• Perform root cause analysis (RCA) and implement preventive measures.
• Create and maintain operational documentation and standard operating procedures.
• Ensure compliance with security, audit, and regulatory requirements.

Required Skills:

• Apache Kafka Administration
• Food and Beverage and MSK
• AWS Cloud Services
• Linux (RHEL/Rocky Linux)
• Shell Scripting and Python
• Jenkins, Git, Ansible, Terraform
• Docker and Kubernetes
• Monitoring Tools (Grafana, Prometheus, Splunk)
• Networking, Security, and High Availability Concepts
• Performance Tuning and Capacity Planning

Preferred Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or related field.
• Experience with Cloudera CDP, AWS MSK, Airflow, and Spark.
• AWS, GCP, Kafka, or Kubernetes certifications.
• Experience supporting large-scale production environments handling petabyte-scale data workloads.

Top 3 responsibilities you would expect the Subcon to shoulder and execute:

• Perform infrastructure provisioning and automation using Infrastructure as Code (IaC).
• Design and maintain CI/CD pipelines using Jenkins, Git Hub Actions, Git Lab CI, or similar tools.
• Automate infrastructure deployment using Terraform, Cloud Formation, and Ansible.

Manage source control repositories and release processes.

Implement monitoring and alerting solutions using Prometheus, Grafana, Splunk, ELK, or Cloud Watch.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.