Senior Data Engineer

Location: Bengaluru

Location :
Bangalore (HYBRID)
Department :
Analytics & Engineering
Team :
Quantitative Analytics

About the Company:

Kamakhya Analytics is a technology-driven research and consulting organisation helping leaders, parties, and institutions decode India’s political and policy landscape. We bring together survey science, data analytics, and strategic insight to answer complex questions about public opinion, governance, and performance.

From ground-level fieldwork to advanced modelling and interpretation, every project is built for clarity, accuracy, and impact. Our teams manage the complete research cycle design, execution, data engineering, and delivery under a single
framework of discipline, transparency, and speed. Quality and integrity define our work. We apply rigorous controls in sampling and verification, maintain secure and reproducible data systems, and communicate results that withstand scrutiny.

At Kamakhya, our mission is simple: turn evidence into strategy and insight into
action.

Role

Summary:

As a Senior Data Engineer (L1) at Kamakhya Analytics, you will build and own reliable, scalable data workflows that power analysis, modelling, reporting, and leadership decision-making. You will integrate data from multiple sources (field, survey, electoral, digital, and internal systems), standardize it into trusted datasets, and ensure teams operate on consistent metric definitions and a clear single source of truth. This is a hands-on role with strong ownership: you will ensure day-to-day pipeline health, implement structured reliability improvements, and set the bar on documentation, review practices, traceability, and governance - especially important given the sensitivity of political and citizen-level data.

Youwill also mentor junior engineers and drive operational discipline as the data function scales.

What You'll Do

● Build production-grade data pipelines that support polling, constituency analytics, modelling, dashboards, and reporting.

● Integrate and harmonize data from multiple sources, ensuring clean joins and consistent identifiers.

● Create trusted datasets and curated marts that teams can use repeatedly without rework - reducing manual effort and improving turnaround time.

● Strengthen governance: access controls, secure sharing, auditability, and compliance-aligned handling of sensitive data.

● Raise engineering standards: code reviews, testing discipline, documentation, runbooks, and reproducible workflows.

● Mentor junior engineers through pairing, design reviews, and structured coaching.

Key Responsibilities Data Platform & Architecture

● Architect, build, and operate scalable, fault-tolerant data pipelines for structured and unstructured data.

● Drive operational excellence: SLAs, monitoring, incident response, and pipeline performance tuning.

● Design and maintain data lakes/warehouses and ELT/ETL frameworks across cloud and hybrid setups.

● Own and evolve AWS-based data architecture,designing secure, scalable pipelines and storage/compute patterns, optimizing cost and performance, and ensuring reliable deployments through monitoring, alerting, and infrastructure best practices. Governance, Security & Quality

● Establish best practices for data lineage, access controls, secure handling, documentation, and observability.

● Implement automated validation checks, anomaly detection, and runbooks; conduct RCA for recurring issues. Stakeholder Partnership

● Partner closely with analysts, application engineers, and strategy/campaign teams to translate business needs into robust datasets and metrics.

● Ensure consistent metric definitions and prevent multiple versions of the truth.

Must-Have Qualifications

● 5+ years of hands-on Data Engineering experience with a proven track record of building and operating production systems.

● Hands-on experience designing and operating scalable, secure data architectures on AWS, including core services such as S3, IAM, Glue, EMR/EKS/ECS, Lambda, Redshift, Cloud Watch, with a strong understanding of VPC networking, security best practices, cost optimization, fault tolerance, and high availability.

● Advanced proficiency in Python, & SQL.

● Strong experience with…