Data engineer Machine learning

Job Title:

Senior Databricks Data Engineer (Machine Learning Mandatory)

Location:

Hyderabad/Pune/Bangalore

Work mode:
Work from Office (4 Days per week)

Experience:

5+ Years

Job Summary

We are seeking a Databricks Data Engineer with strong expertise in Azure Data Factory (ADF), PySpark, SQL, and Machine Learning. The ideal candidate must have hands-on experience building data pipelines and working directly on Machine Learning projects, including data preparation, feature engineering, model deployment support, and ML lifecycle management.

Key Responsibilities

- Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and Azure Data Factory (ADF).
- Build and optimize ETL/ELT processes for large-scale data processing.
- Develop complex SQL queries, data models, and performance optimizations.
- Prepare, cleanse, and transform data for Machine Learning use cases.
- Implement feature engineering and ML data pipelines in Databricks.
- Work with MLflow and Databricks Machine Learning capabilities for experiment tracking and model management.
- Ensure data quality, governance, security, and reliability across data platforms.

Mandatory Skills

- Strong hands-on experience with Databricks and PySpark.
- Expertise in Azure Data Factory (ADF).
- Advanced SQL skills for data transformation and optimization.
- Hands-on Machine Learning experience is mandatory, including:
- Data preprocessing and feature engineering.
- Model training and evaluation.
- ML pipeline development.
- Model deployment and monitoring.

- Experience with MLflow and Databricks ML.

- Experience with Azure Data Lake, Delta Lake, and cloud-based data platforms.
- Proficiency with Git and CI/CD practices.

Regards,

Anil Kumar