Big Data Developer Python - Raritan, NJ; Onsite

Big Data Developer with Python Experience - Raritan, NJ (Onsite day
1)

Experience on Bigdata Hadoop ecosystem, Data lakes, DWH, structured/Unstructured Data, creating Data pipeline/Data frames, Data validations, Querying Data bases using SQL.

• Development, customize and manage integration tools, databases, warehouses and analytical systems with the use of data related instruments/instances
• Create and run complex queries and automation scripts for operational data processing and building out Python ETL processes and writing complex SQL queries.
• Test the reliability and performance of each part of a system and cooperate with the testing team
• Deploying data models into production environments. This entails providing the model with data stored in a warehouse or coming directly from sources, configuring data attributes, managing computing resources, setting up monitoring tools, etc.
• Responsible for setting up tools to view data, generate reports, and create visuals
• Monitoring the overall performance and stability of the system. Adjust and adapt the automated pipeline as data/models/requirements change.
• Excellent understanding of ETL cycle. Analyze and organize raw data, build data systems and pipelines.
• Combine raw information from different sources, explore ways to enhance data quality and reliability, interpret trends and patterns from the raw data.
• Experience in using of Python/ PySpark and/or Scala for data engineering.
• Understanding of data types/ handling of different data models.
• Good knowledge in various phases of SDLC Requirement Analysis, Design, Development and Testing on various Development and Enhancement Projects.
• Desirable to have experience with Spark, Flink, Kafka, Flask, Scala, PySpark for Data engineering.
• Experience with the Microsoft Azure or AWS data management tools such as Azure Data factory, Datalake and Databricks or AWS Snowflake is a plus
• Experience with data visualization tools is a plus (Power
  
  BI, Tableau).
• Understanding of descriptive and exploratory statistics, predictive modelling, evaluation metrics, decision trees, machine learning algorithms is a plus
• Good scripting and programming skills.

Primary Skill: CSST, Data Extraction, JDK, JSTL.

• Seniority level:
  Mid-Senior level
• Employment type:
  
  Full-time
• Job function:
  Other
• Industries: IT Services and IT Consulting