Senior Data Engineer

Location: Bengaluru

As a Senior Lead/Staff Data Engineer, you will play a pivotal role in architecting, building, and maintaining large‑scale data platforms and pipelines. You will work across distributed systems, large datasets, and cloud-native environments to enable advanced analytics, real-time insights.

In this role, you will design and lead the development of robust data solutions using both Python and Java, collaborate with cross‑functional data, platform, and product teams, and guide engineering best practices across the organization. You will also mentor and empower engineers, ensuring delivery of high‑quality, scalable, and secure data services.

You will be responsible

Leading the end-to-end design and implementation of highly scalable data pipelines, data services, and distributed processing systems using Python and Java. Experience in Migrating large scale data Architecting batch, micro-batch, and real-time data processing solutions using Spark, Kafka, Flink, and cloud-native services.

Owning and improving the data platform’s reliability, performance, security, and cost efficiency.

- Driving best practices in coding, architecture, CI/CD, testing, observability, and infrastructure-as-code.
- Partnering with product, security to deliver business-critical data solutions.
- Leading incident resolution, performance tuning, and platform optimization efforts.
- Mentoring junior engineers, promoting engineering excellence, and contributing to technical roadmaps.
- Supporting the adoption of ML/AI and cybersecurity-oriented data capabilities.

You will n

- need 9+ years of hands‑on experience in Data Engineering, with at least 2+ years in a lead/senior technical role.
- Expert programming skills in Python and Java, including experience building distributed data applications, APIs, microservices, and automation.
- Strong command of SQL and advanced data modelling for analytical and operational workloads.
- Hands-on experience building cloud-scale data platforms using:
Databricks, Azure Data Lake, or AWS/GCP equivalents.
- Deep understanding of big data technologies such as Spark, Kafka, Flink, Hadoop ecosystem.

- Experience with Kubernetes, Docker, CI/CD pipelines, Infrastructure-as-Code (Terraform), and reusable modules.
- Working knowledge of ETL/ELT frameworks and orchestration tools (Airflow, dbt, ADF, Glue, Dagster, or Prefect)
- ETL Tools:
Oracle Data Integrator, Talend, Fivetran.
- Ability to drive clarity, lead architectural discussions, and empower junior team members.
- Familiarity with machine learning and AI tools (desirable).
- Understanding the IFS ETL tool would be preferrable.
- Understanding of cybersecurity principles and secure data engineering practices.