Azure Data Engineer

Location: Bengaluru

Azure Data Engineer: 5+ years

Data Management and Storage:
Design and implement data storage solutions using Azure SQL Database, Azure Data Lake Storage and Azure Blob Storage.
Ensure data storage systems are scalable, performant, and cost-effective.
Data Integration and ETL:
Develop and manage ETL (Extract, Transform, Load) processes using Azure Data Factory.
Ensure efficient data cleaning, transformation, and loading into Data Lakes.
Big Data and Analytics:
Utilize big data technologies like Azure Databricks and Apache Spark for data processing and analytics.
Create data processing workflows and pipelines to support data analytics and machine learning applications.
Data Pipeline Architecture:
Design, construct, and maintain data pipelines and combine large, complex datasets that meet business requirements.
Optimization and Monitoring:
Implement effective metrics and monitoring processes to ensure data quality and system performance.
Identify and implement internal process improvements, such as automating manual processes and optimizing data delivery.
Collaboration and Support:
Work with stakeholders to address data-related technical issues and support data-driven decision-making.
Collaborate with data and analytics experts to enhance data system functionality.
ML Use Cases:
Collaborate with data scientists to create datasets for variety of use cases .
Implement data annotations, curation techniques to enhance data readiness for ML Applications.