Senior Data Engineer

Location: Vasanthanagar

Data Engineer – Azure Databricks

Experience:

6+ Years

Location:

Bengaluru
- Whitefield
Work Mode:  Hybrid – 3 days/week
Interview Process:  2 Technical Round

Job Summary

We are looking for an experienced  Data Engineer  with 6+ years of hands-on experience in building and managing scalable data engineering solutions. The ideal candidate should have strong expertise in  Spark, Scala, Airflow, Hive, Kubernetes, Iceberg, and Azure Databricks .
The candidate should also have a good understanding of modern data platforms, cloud technologies, CI/CD practices, and the ability to leverage  AI across the Software Development Lifecycle (SDLC) and Data Engineering  to drive innovation and business value.

Key Responsibilities
Design, develop, and maintain scalable and reliable data engineering pipelines.
Build data processing solutions using  Apache Spark and Scala .
Develop and manage workflows using  Apache Airflow .
Work with  Hive, Iceberg, Kubernetes, and Docker  in modern data platforms.
Develop and maintain CI/CD pipelines using  Git Hub Actions .
Design and implement data solutions using  Azure Databricks .
Work with  ADLS, Azure Data Factory (ADF), Delta Lake, Databricks Jobs and Pipelines .
Utilize  Lakeflow Connect, Unity Catalog, and Databricks Compute (Classic & Serverless) .
Implement data governance, security, and access controls using Unity Catalog.
Optimize data pipelines and processing workloads for performance and scalability.
Collaborate with data scientists, architects, product teams, and other engineering teams.
Apply AI capabilities and tools across the SDLC and Data Engineering lifecycle to improve productivity, automation, and business outcomes.
Follow engineering best practices around version control, testing, deployment, monitoring, and documentation.

Required Skills
Must-Have – Priority 1
Strong hands-on experience with  Apache Spark
Scala
Apache Airflow
Hive
Kubernetes
Apache Iceberg
Good knowledge of  Docker
Good experience with  Git Hub Actions / CI-CD
Must-Have – Priority 2
Strong experience with  Azure Databricks
ADLS
Databricks Jobs & Pipelines
Azure Data Factory (ADF)
Delta Lake
Lakeflow Connect
Unity Catalog
Databricks Compute –  Classic & Serverless
Must-Have – Priority 3
Experience using  AI tools/capabilities within SDLC and Data Engineering
Ability to identify opportunities to use AI for automation, development, optimization, and innovation
Strong problem-solving and analytical skills

Additional Requirements
6+ years of relevant experience in Data Engineering.
Strong understanding of data engineering concepts and distributed data processing.
Excellent communication and collaboration skills.
Candidates must be based in or willing to relocate to  Bengaluru .
Candidates should be comfortable working from the office
Willingness to attend  2 technical interview rounds .

Preferred Candidate Profile

Candidates with strong hands-on experience across  Spark/Scala/Airflow  and the  Azure Databricks ecosystem  will be given priority. Experience in modern data lakehouse technologies, containerization, CI/CD, and practical application of AI in engineering environments will be an added advantage.