Big Data Developer

Job Title

Senior Big Data

St. Louis, MO (Onsite)

1 year w/ extension

Contract (W2)

$50/hr

We are seeking a highly experienced Senior Big Data / AWS Administrator with 10+ years of hands‑on experience in managing large-scale distributed systems. The ideal candidate will have deep expertise in AWS cloud infrastructure, Hadoop ecosystem, and Kafka administration, ensuring high availability, scalability, and performance of enterprise data platforms.

• Administration, Hadoop, HDFS, Hive, Spark

• Design, deploy, and manage scalable, secure, and highly available infrastructure on AWS.
• Implement automated infrastructure provisioning using Infrastructure as Code (Terraform/Cloud Formation).
• Optimize cost, performance, and reliability of cloud environments.
• Develop disaster recovery and backup strategies.
• Install, configure, tune, and maintain Hadoop ecosystem components (HDFS, YARN, Hive, HBase, Spark).
• Monitor cluster health, performance, and capacity planning.
• Troubleshoot cluster issues and ensure high availability using HA configurations (Name Node HA, etc.).
• Manage data ingestion and processing workflows.
• Install, configure, and manage Apache Kafka clusters.
• Ensure high throughput, low latency, and fault‑tolerant streaming systems.
• Manage topics, partitions, brokers, and retention policies.
• Perform Kafka cluster scaling, upgrades, and performance tuning.
• Implement monitoring using tools like Prometheus, Grafana, or Confluent Control Center.
• Implement monitoring, logging, and alerting for all systems.
• Conduct root cause analysis and performance tuning.
• Ensure system security, access controls, and compliance standards.
• Maintain detailed documentation and operational runbooks.
• Build CI/CD pipelines for infrastructure and platform deployments.
• Collaborate with Dev Ops and engineering teams for platform improvements.

• Minimum 10+ years of experience in system administration, with at least 5+ years in big data and cloud environments.
• Proven experience managing large-scale Hadoop clusters and Kafka environments.
• Strong hands‑on experience with AWS cloud platform.