Data Engineer

* Data Engineer# Data Engineer
** Verinon Technology Solutions Ltd.
** has an opening for the position of
** Data Engineer** *(Job Code DE)*.
*
* Work Location:

** Arlington Heights, IL and relocation required to various unanticipated locations throughout the U.S.
*
* Job Duties:

** Responsible for data engineering including cloud, data processing, and data analytics deployment and integration.  Analyze user needs and define, design, and test using Azure Databricks and ETL technologies.  Extract, transform, and load (ETL) data from various source systems to Azure data storage services using Azure Databricks, Azure Data Factory, PySpark, and Snowflake for visualization and analysis.  Perform data ingestion to Azure Data Lake Storage (ADLS) and Delta Lake tables and processing data in Azure Databricks.  

Conduct tuning of SQL queries.  Create pipelines, data flows, and complex data transformations.  Analyze existing databases, tables, and other objects to migrate data.  Use Python Pandas for data cleansing and validating the source data.  Implement different design patterns to mask and generate test data as per functional test needs.  Create and process numerous masking rules to protect sensitive data.  Migrate data from production systems to create copies and provision to lower environments.  

Create different masking rules as per the enterprise needs to mask data across different databases. Schedule meetings with senior data leads and analyze the data.  Work closely with data privacy and information security team to secure production data copied into non-production environments by masking sensitive data elements as defined by privacy standards.  Identify columns in the table by running the data profiler and mapping the masking algorithms using rule sets.  

Implement data scanning and catalog of solutions.  Troubleshoot configuration and connectivity issues within platforms.  Use of Azure Data Factory, Azure Databricks, PySpark, Snowflake, BigID, Delphix, Azure SQL, Python, SQL, Oracle, and SQL Server.
** Requirements:
** This position requires a Master’s degree or foreign academic equivalent in Computer Science, Computer Engineering, Information Systems, Information Technology, or a related field.  Plus 2 years of experience in the job offered or a related occupation including Data Consultant or related experience.  Position requires 2 years of experience with Azure Data Factory, Azure Databricks, PySpark, Azure SQL, and Python.  40hrs/week.
** Salary:**  $ 119,558 / year
** Application Instructions:
** To apply, please MAIL resume to referenced address, ensuring to indicate the position reference code *(Job Code DE)
* for which you are applying.  Mail to:  Verinon Technology Solutions Ltd., Attn:
Vice President, 3395 North Arlington Heights Rd., Arlington Heights, IL 60004.## Apply for this position
#J-18808-Ljbffr