Hadoop Data Engineer
Listed on 2026-02-07
-
IT/Tech
Data Engineer
Overview
Must Have Technical/Functional Skills Primary Skill:
Hadoop (HDFS, Hive, Spark), Big Data ETL/ELT, Distributed Processing, MS SQL Server
Secondary: Artificial Intelligence/ Machine learning
Experience: Minimum 10 years
Responsibilities- Design and implement scalable batch and/or streaming data pipelines using Hadoop ecosystem tools.
- Develop and optimize data ingestion processes from multiple sources (RDBMS, files, APIs, logs).
- Build and maintain datasets in HDFS/Hive and ensure data quality, lineage, and governance.
- Perform performance tuning for distributed workloads (partitioning, file formats, resource management).
- Create and optimize complex queries, stored procedures, and ETL workflows in MS SQL Server.
- Collaborate with data scientists/analysts to deliver feature-ready datasets for ML models.
- Implement monitoring and alerting for pipeline health and data SLAs.
- Document architecture, workflows, data dictionaries, and operational runbooks.
- Support production deployments, incident triage, and root cause analysis.
Skills & Qualifications
- Strong hands-on experience with Hadoop components (e.g., HDFS, Hive, YARN, Map Reduce/Spark).
- Experience with data modeling and data warehousing concepts.
- Solid proficiency in MS SQL Server (T-SQL, query optimization, indexing, stored procedures).
- Experience with ETL/ELT design patterns and job scheduling (e.g., Oozie/Airflow/Control-M).
- Strong understanding of distributed computing concepts and performance tuning.
- Familiarity with Python/Scala/Java for data processing (any one preferred).
- Bachelor’s degree in Computer Science, Engineering, or equivalent experience.
Discretionary Annual Incentive.
Comprehensive Medical Coverage:
Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
Family Support:
Maternal & Parental Leaves.
Insurance Options:
Auto & Home Insurance, Identity Theft Protection.
Convenience & Professional Growth:
Commuter Benefits & Certification & Training Reimbursement.
Time Off:
Vacation, Time Off, Sick Leave & Holidays.
Legal & Financial Assistance:
Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
Salary Range: $110,000 - $125,000 a year
Qualifications: BACHELOR OF COMPUTER SCIENCE
#J-18808-Ljbffr(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).