Big Data Engineer; Cloudera, AWS

Position: Big Data Engineer (Cloudera, AWS)
Job Title: Big Data Engineer (Cloudera, AWS)

Location: Primarily Remote/ HYB (Monthly Onsite Meetings in Reston, VA and/or Washington, DC)
Employment Type: Contract

GC/USC

8+ years of experience in Cloudera and AWS Cloud environments with strong expertise in Kafka, NiFi, HBase, Solr, Mongo

DB, automation scripting, and performance optimization for large-scale enterprise data systems.

Onsite interview needed
Overview:

Big Data Engineer with strong Cloudera and AWS expertise to support a high-availability Production Support Environment. This role involves managing big data infrastructure, optimizing Cloudera components, and ensuring seamless data migrations and automation. Occasional weekend releases and late-night troubleshooting may be required.
Key Responsibilities:

• Maintain and optimize Cloudera environments (CDP, CDSW, Hive, Spark, NiFi).
• Perform critical data migrations from CDH to CDP.
• Design and implement Big Data pipelines and automated data flows using Python/R and Ni Fi .
• Automate project lifecycle processes and perform incremental upgrades.
• Support analytics, ML, and data science use cases across infrastructure and cluster deployments.
• Manage data governance, quality, testing, and documentation.
• Use YARN for resource allocation and job scheduling.
• Manage workflows via Oozie and Hue.
• Enforce security policies across Hadoop clusters using Apache Ranger.
• Configure and manage Cloudera Data Science Workbench via Cloudera Manager.
• Troubleshoot Kerberos, TLS/SSL, and data science workload issues.
• Provide end-to-end Big Data / Hadoop support throughout development and production.
• Optimize performance, replication, and connectivity; implement monitoring and automation scripts.
• Support and manage Kafka, Ni Fi , HBase, Solr, and Flume integrations.

Required Skills & Experience:

• 8+ years of experience supporting Cloudera applications in AWS Cloud environments.
• Expertise with MongoDB, Kafka, Ni Fi , Flume, HBase, and Solr.
• Strong knowledge of Cloud Disaster Recovery strategies.
• Kafka Administration & Troubleshooting (brokers, Connect, Zookeeper, MQ integration).
• Experience with Kafka Streams API, KStreams, and KTables.
• NiFi Administration (flow management, registry server, controllers, service management).
• Integration experience: NiFi to Kafka/HBase/Solr, Flume to Kafka/NiFi/MQ.
• Strong skills in database management, query optimization, and automation scripting.
• Hands-on experience supporting large-scale enterprise data environments.

Preferred Skills:

• Experience with DB2 UDB, DB2 mainframe, or MongoDB.
• Exposure to streaming technologies such as Kafka, Spark, and Kudu.
• Familiarity with AI/ML workloads and GenAI testing environments.