More jobs:
Data Engineer
Remote / Online - Candidates ideally in
Morris Plains, Morris County, New Jersey, 07950, USA
Listed on 2025-12-27
Morris Plains, Morris County, New Jersey, 07950, USA
Listing for:
Highbrow LLC
Remote/Work from Home
position Listed on 2025-12-27
Job specializations:
-
IT/Tech
Data Engineer, Cloud Computing
Job Description & How to Apply Below
Job Title:
Data Engineer
Job :
Job Location: Morris Plains, NJ (remote to start)
Job Travel Location(s):
# Positions: 2
Employment Type: W2
Candidate Constraints:
Duration: Long Term
# of Layers: 0
Work Eligibility: All Work Authorizations are Permitted
Key Technology: Spark, Py-Spark, Shell scripting, Teradata, Hive and Hadoop
Job Responsibilities- Work with business and technical leadership to understand requirements.
- Design to the requirements and document the designs.
- Ability to write product-grade performant code for data extraction, transformations and loading using Spark, Py-Spark
- Do data modeling as needed for the requirements.
- Write performant queries using Teradata SQL, Hive SQL and Spark SQL against Teradata and Hive
- Implementing dev-ops pipelines to deploy code artifacts on to the designated platform/servers like AWS or Hadoop Edge Nodes
- Implement Hadoop job orchestration using Shell scripting, Apache Oozie, CA7 Enterprise Scheduler and Airflow
- Troubleshooting the issues, providing effective solutions and jobs monitoring in the production environment
- Participate in sprint planning sessions, refinement/story-grooming sessions, daily scrums, demos and retrospectives.
- Strong development experience in Spark, Py-Spark, Shell scripting, Teradata, Hive and Hadoop
- Experience of Ab Initio is a bonus.
- Strong experience in writing complex and effective SQLs (using Teradata SQL, Hive SQL and Spark SQL) and Stored Procedures
- Excellent work experience on Hadoop as data warehouse/Data Lake implementations
- Experience in Agile and working knowledge on Dev Ops tools (Git, Jenkins, Artifactory)
- Unix/Linux Shell scripting (KSH) and basic administration of Unix servers
- CA7 Enterprise Scheduler
- Experience with AWS (S3, EC2, SNS, SQS, Lambda, ECS, Glue, IAM, and Cloud Watch)
- Databricks (Delta lake, Notebooks, Pipelines, cluster management, Azure/AWS integration)
- Experience in Jira and Confluence
- Exercises considerable creativity, foresight, and judgment in conceiving, planning, and delivering initiatives.
- Health care domain knowledge is a plus.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×