Actively ETL Data Engineer

Actively Looking for a ETL Data Engineer

Location:

Jersey City, NJ

Duration:

18

Months+ Extension

Hourly Rate:

Depending on Experience (DOE)

Work Authorization:

Location:

Jersey City, NJ/ NY onsite Day 1 (need only Locals)

Please don t share Fake Consultants

Mail:

Job Description:

• Good understanding of ETL, BI and DW technologies.
• Hands-on experience on JSON or XML files as sources.
• Experience in autoloader for incremental loading and explode options for flattening data
• Experience in loading Delta live tables
• Experience in PySpark or Spark using data frames.
• Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure.
• Work together with data scientists and analysts to understand the needs for data and create effective data workflows.
• Create and maintain data storage solutions including Azure SQL Database, Azure Data Lake, and Azure Blob Storage.
• Utilizing Azure Databricks to create and maintain ETL (Extract, Transform, Load) operations.
• Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data.
• Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
• Monitoring and resolving data pipeline problems to guarantee consistency and availability of the data.

AI TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. AI TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters.

Nor will AI TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract