Gen AI Engineer

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory, Synapse Pipelines, and Databricks.
• Ingest, transform, and curate structured and unstructured data from diverse sources for AI/ML consumption.
• Optimize data workflows for performance, reliability, and cost-efficiency.
• Enable data access and preparation for Azure AI services including Azure Machine Learning, Azure OpenAI, and Cognitive Services.
• Collaborate with data scientists and ML engineers to operationalize AI models using MLOps and GenAIOps practices.
• Support RAG (Retrieval-Augmented Generation) and other GenAI patterns through data engineering best practices.
• Implement CI/CD pipelines for data workflows using Azure Dev Ops or Git Hub Actions.
• Automate infrastructure provisioning using Terraform, Bicep, or Client templates.
• Monitor and troubleshoot data pipelines and cloud resources using Azure Monitor and Log Analytics.

• 8-10+ years of experience in data engineering, cloud platforms, and big data technologies.
• Strong proficiency in Azure services:
  Data Factory, Synapse, Data Lake, Azure SQL, Cosmos DB.
• Experience with Python, Spark, SQL, and distributed data processing.
• Familiarity with AI/ML workflows and tools such as Azure ML, MLflow, and OpenAI APIs.
• Understanding of data governance, security, and compliance in cloud environments.