Hadoop Developer

Key Responsibilities

• Design, develop, and implement scalable big data solutions using Hadoop
  .
• Work with Hadoop ecosystem tools such as HDFS
  , Map Reduce
  , YARN
  , Hive
  , Pig
  , HBase
  , and Spark
  .
• Develop and optimize ETL pipelines for large datasets.
• Write and optimize HiveQL queries for data analysis and reporting.
• Integrate Hadoop systems with data sources like RDBMS
  , No
  
  SQL databases
  , and streaming systems.
• Monitor Hadoop cluster performance and troubleshoot issues.
• Ensure data security, governance, and compliance.
• Collaborate with data engineers, data scientists, and business teams.
• Document system designs, workflows, and best practices.

• Strong experience with Hadoop framework and ecosystem
  .
• Proficiency in Java
  , Scala
  , or Python
  .
• Hands-on experience with Hive
  , Pig
  , HBase
  , Spark
  , and Sqoop
  .
• Knowledge of Linux/Unix environments.
• Experience with SQL and No
  
  SQL databases.
• Understanding of distributed systems and data processing concepts
  .
• Familiarity with data warehousing and ETL tools
  .

• Experience with Apache Spark and real-time processing tools like Kafka or Flume
  .
• Knowledge of cloud platforms (AWS EMR, Azure HDInsight, or Google Dataproc).
• Understanding of Dev Ops tools (Git, Jenkins, CI/CD pipelines).
• Hadoop certification (Cloudera, Hortonworks) is a plus.

• Bachelor’s degree in Computer Science, Engineering, or related field
  .
• 8 years of experience in Big Data or Hadoop development (varies by role level).