Big Data Platform Engineer

Big Data Platform Engineer.

Jersey City, NJ (Onsite).

Contract, 12 months.

Zip Staff is hiring a Big Data Platform Engineer for our client, a leading global financial services firm. This 12-month contract role focuses on managing, supporting, and optimizing enterprise-grade big data platforms, with deep hands‑on expertise in Apache Spark, HDFS, and Kafka as the primary focus. You will administer production Spark environments, HDFS clusters, and Kafka infrastructure, including monitoring, performance tuning, security, and lifecycle operations.

Experience administering these technologies within a Cloudera environment is preferred but secondary to strong expertise in the underlying platforms. Candidates must be local to the Jersey City, NJ area; this is a fully onsite role.

• Deploy, configure, monitor, tune, and troubleshoot Apache Spark environments and workloads, including job performance, resource management, capacity planning, and production operations.
• Administer HDFS for capacity, performance, availability, permissions, replication, and troubleshooting, with strong understanding of HDFS architecture and integration with distributed compute platforms.
• Deploy, configure, manage, monitor, and troubleshoot Kafka clusters, topics, producers, and consumers, covering performance, capacity, availability, security, and production operations.
• Monitor and tune platform performance using Grafana, Splunk, and related tools, and resolve bottlenecks across Spark, HDFS, and Kafka.
• Implement and maintain security and compliance controls, including Kerberos, SSL/TLS, authorization policies, HDFS ACLs, and enterprise IAM/LDAP integration.
• Automate operational tasks using Python, Bash, or Perl.
• Provide expert troubleshooting and production support, including participation in an on‑call rotation.
• Perform platform administration including installs, patching, upgrades, migrations, and lifecycle management with minimal downtime;
  Cloudera Hadoop/CDP administration is a secondary focus.

• 3+ years of hands‑on production administration experience with Apache Spark, HDFS, and Kafka.
• Strong demonstrable expertise in Spark administration, tuning, and troubleshooting.
• Strong demonstrable expertise in HDFS administration, including storage management, security, and performance.
• Strong demonstrable expertise in Kafka administration, including topic management, producer/consumer architecture, performance, and security.
• Strong Linux and command‑line skills.
• Experience with Grafana and Splunk for monitoring and performance analysis.
• Solid knowledge of Kerberos, SSL/TLS, and enterprise IAM/LDAP.
• Excellent problem‑solving, analytical, and communication skills.

• Experience administering Spark, HDFS, and Kafka within a Cloudera Hadoop/CDP environment, including Cloudera Manager.
• Cloudera or equivalent certifications.
• Automation experience with Python, Bash, or Perl.

• Health Insurance.
• Dental Insurance.
• Vision Insurance.
• 401(k) + matching.
• Paid time off.
• Referral Program.

Zip Staff connects skilled technology professionals with leading organizations across the United States, matching top talent with impactful contract and full‑time opportunities.