ETL Lead

Data Conversion & Migration Experience with experience in Azure Services (Azure Data Factory, Azure Data Bricks, Azure BLOB) , Python Scripting

• 8+ Years of experience in ETL
• 5+ Years of experience in Azure Data Factory is a must
• 5+ Years of experience in Azure Databricks is a must
• Experience in writing Python scripts using Databricks is a must
• Hands on experience in migration projects on Azure Cloud.
• Should be able to evaluate and provide the best ETL tool as per data migration requirement
• Hands on experience in migration project using Azure Data Factory from SQL Server, DB2 (LUW or Mainframe), Mainframe database (VSAM, flat files, DB2) to Oracle (If do not have experience in this, should have experience from any relational source to cloud database using ADF)
• Experience on validating Data migration
• Hands on experience in data extraction (full data + incremental data) strategies from source systems. Source systems like relational database – SQL Server, DB2, mainframe database – VSAM files / flat files / Sequential files
• Hands on experience in implementing bi-directional synchronization and what database configurations need to be done in the database to enable bi-directional sync
• Documents all technical and system specifications documents for all ETL processes and perform unit tests on all processes and prepare required programs and scripts.
• Design, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities
• Gather requirements and business process knowledge in order to transform the data in a way that's geared towards the needs of end users
• Maintain and improve already existing processes
• Ensure that the data architecture is scalable and maintainable
• Work with the business in designing and delivering correct, high quality data
• Investigate data to identify potential issues within ETL pipelines, notify end-users and propose adequate solutions