Hadoop Data Engineer

Job Title

Hadoop Data Engineer

Location: Chicago, IL

Work Arrangement: Onsite

Client Industry: Financial Services

Duration: 12 Months Contract

Schedule: Standard Hours (Monday - Friday, 8 AM to 5 PM)

Seniority level: Entry level

Employment type: Contract

Job function: Information Technology

• Design, develop, and maintain batch and near‑real‑time data pipelines using Spark Structured Streaming, Map Reduce, and other Big Data frameworks.
• Ingest data from multiple sources such as message queues (Kafka), file shares, REST APIs, and relational databases.
• Transform, clean, and validate data in HDFS, Hive, Impala, or Spark SQL.
• Convert and manage data in formats like JSON, CSV, XML to support downstream analytics.
• Perform data validation, profiling, and analysis to identify anomalies and ensure data integrity.
• Troubleshoot issues in data pipelines, SQL jobs, or Spark applications, including slow‑running jobs or failures.

• Strong SQL skills – MySQL, Hive, Impala, or Spark SQL.
• Experience ingesting data from message queues, file shares, REST APIs, relational databases, and handling formats such as JSON, CSV, XML.
• Experience working with Spark Structured steaming.
• Experience with Hadoop/Big Data and distributed systems.
• Hands‑on programming experience in at least one of Scala, Python, or PHP.
• Working knowledge of Spark, Sqoop, Kafka, Map Reduce, No
  
  SQL databases (e.g., HBase, Solr, CDP, HDP, Cloudera, Hortonworks), Elasticsearch, Kibana.

• Prior bank experience.
• Experience with Service Now.

Hourly Rate: $60 – $65 per hour

This reflects base compensation and may vary based on candidate qualifications.

The Company offers the following benefits for this position, subject to applicable eligibility requirements: medical insurance, dental insurance, vision insurance, 401(k) retirement plan, life insurance, long‑term disability insurance, short‑term disability insurance, paid parking/public transportation, paid time off, paid sick and safe time, hours of paid vacation time, weeks of paid parental leave, and paid holidays annually – as applicable.

At Collabera, we build careers by providing opportunities to work with top organizations, cutting‑edge technologies, and dynamic teams. Join a culture that thrives on innovation, collaboration, and a commitment to excellence.