Senior Data Engineer

Data Engineer – Databricks

Experience:

7+ Years
Work Mode:  Remote
Shift Timing:  12:30 PM to 9:30 PM IST

Primary

Skills:

Databricks, Pyspark, Python, SQL, Microsoft Fabric, Azure Data Factory, Data Ops practices, and performance optimization

Job Summary:

We are looking for experienced  Data Engineers  with strong hands-on expertise in  Databricks, PySpark, Python, SQL, and big data processing . The ideal candidate will have proven experience designing and developing scalable data ingestion pipelines, analytical data models, and high-performance data solutions.

Experience with  Microsoft Fabric, Azure Data Factory, Data Ops practices, and performance optimization  will be an added advantage.

Key Responsibilities:

Design, develop, and maintain scalable  data engineering pipelines  using Databricks and PySpark.
Develop robust  data ingestion and transformation pipelines  for batch and large-scale data processing.
Work with  Databricks  for data processing, transformation, orchestration, and analytics.
Develop efficient and optimized  PySpark and Python  code for big data workloads.
Write complex  SQL queries  for data transformation, validation, analysis, and troubleshooting.
Design and implement  analytical data models  to support reporting and business analytics.
Build and maintain data pipelines using  Azure Data Factory (ADF) .
Implement data quality, validation, monitoring, and error-handling mechanisms.
Apply  Data Ops practices  to improve data pipeline reliability, automation, deployment, and operational efficiency.
Identify and resolve performance bottlenecks across data pipelines, queries, Spark jobs, and Databricks workloads.
Collaborate with data architects, analysts, application teams, and other stakeholders to deliver scalable data solutions.
Follow best practices for data security, governance, testing, version control, and CI/CD

Required Skills:

7+ years of experience  in Data Engineering.
Strong hands-on experience with  Databricks .
Strong expertise in  PySpark  and big data processing.
Strong programming skills in  Python .
Advanced  SQL  skills.
Experience developing  data ingestion and ETL/ELT pipelines .
Experience in  analytical data modeling  and data transformation.
Good understanding of distributed data processing and scalable data architectures.
Strong troubleshooting and performance optimization skills.

About Atain:

()
Atain is an enterprise orchestration partner that aligns people, processes, and platforms with AI to redefine work and deliver assured outcomes. It is enabled through the SMART Framework driving superior metrics with AI-redefined work, the Tech Bud platform, a GenAI foundation built on zero-pilot architecture and ISO 42001 standards, a robust partner ecosystem, and deep domain expertise. Atain's Unified Hub continuously senses, predicts, and re-orchestrates work across people, processes, and platforms to deliver assured outcomes.

The centers of skill and scale combine deep expertise with AI to deliver assured outcomes. For more information, visit

Atain is ISO 27001:2013, CMMI SVC Level 5 and ISAE-3402 compliant for IT, and COPC Certified v6.0, ISO 27001:2013 and PCI DSS 3.2 certified for BPO processes. The organization follows Six Sigma rigor for process improvements.

It is our policy to provide equal employment opportunities to all individuals based on job-related qualifications and ability to perform a job, without regard to age, gender, gender identity, sexual orientation, race, color, religion, creed, national origin, disability, genetic information, veteran status, citizenship or marital status, and to maintain a non-discriminatory environment free from intimidation, harassment or bias based upon these grounds.