MLOps Engineer; Databricks​/AWS

Databricks Machine Learning Engineer

Strong hands-on experience with Databricks Machine Learning

Proficiency in Python, PySpark, and SQL for developing and operationalizing machine learning solutions.

Hands-on experience with MLflow for experiment tracking, model registry, model versioning, and model lifecycle management.

Experience deploying and managing machine learning models using Databricks Model Serving and batch inference pipelines.

Strong understanding of the end-to-end ML lifecycle, including feature engineering, model training, validation, deployment, monitoring, and retraining.

Experience building scalable ML pipelines using Databricks Workflows and Delta Lake.

Hands-on experience with AWS services such as S3, IAM, EC2, Lambda, ECR, ECS/EKS, Cloud Watch, and Secrets Manager.

Experience implementing CI/CD pipelines for Databricks and ML workloads using Git, Bitbucket, Jenkins, and Databricks Asset Bundles (DAB).

Experience with infrastructure automation using Terraform (Infrastructure as Code).

Strong understanding of Apache Spark architecture, optimization, and distributed data processing.

Experience working with Unity Catalog for governance, security, and access management.

Knowledge of model monitoring, data drift detection, model performance monitoring, and automated retraining strategies.

Understanding of MLOps best practices, including reproducibility, versioning, testing, and governance.

Strong collaboration skills to work with Data Scientists, Data Engineers, and Platform Engineering teams.

Excellent analytical, problem-solving, and communication skills.

Design, develop, and maintain end-to-end MLOps pipelines on Databricks running on AWS.

Build and automate machine learning workflows covering data preparation, feature engineering, model training, evaluation, deployment, and monitoring.

Deploy and manage ML models using MLflow Model Registry and Databricks Model Serving.

Develop and maintain CI/CD pipelines for ML solutions across development, staging, and production environments.

Collaborate with Data Scientists to product ionize machine learning models and ensure reliable deployments.

Monitor model health, prediction quality, data drift, and system performance, and implement retraining strategies where required.

Optimize Databricks workloads for performance, scalability, and cost efficiency.

Implement Infrastructure as Code (Terraform) for provisioning and managing Databricks and AWS resources.

Ensure platform security, governance, and compliance using Unity Catalog and AWS IAM.

Troubleshoot production issues, perform root cause analysis, and continuously improve platform reliability.

Document MLOps processes, deployment standards, and operational best practices.