Sr. Big Data Hadoop

Experience

5+ years of hands-on experience in Big Data technologies

Key Skills

Big Data Technologies:
Hadoop ecosystem (HDFS, YARN, Map Reduce)
Data Processing:
Apache Spark (Spark SQL, Spark Core)
Querying & Warehousing:
Hive
ETL & Data Pipelines:
Strong experience in ETL design, development, and optimization
Databases & Querying:
Advanced SQL (complex joins, subqueries, performance tuning)

Roles & Responsibilities

Design, develop, and maintain scalable Big Data solutions using Hadoop and Spark
Build and optimize ETL pipelines to process large volumes of structured and unstructured data
Develop and tune Spark jobs for batch and/or near real-time processing
Write efficient Hive queries for data analysis and reporting
Perform data validation, cleansing, and transformation
Optimize query performance and improve data processing efficiency
Collaborate with data analysts, data scientists, and business teams to understand data requirements
Ensure data quality, reliability, and scalability of data platforms
Troubleshoot production issues and provide performance improvements

Required Qualifications

Strong understanding of Big Data architecture and distributed systems
Hands-on experience with Hadoop, Spark, Hive
Excellent SQL skills and data modeling knowledge
Experience working with large-scale datasets
Good understanding of ETL concepts and data warehousing

Good to Have (Optional)

Experience with Kafka / Flume / Sqoop
Exposure to Cloud platforms (AWS, Azure, or GCP)
Knowledge of Python or Scala

Experience with scheduling tools like Airflow / Oozie

Skills:

sql,etl,big data,hive,pyspark,spark,hadoop