Senior Data Engineer

Location: Bengaluru

JD &

Role Overview :

As a Senior Data Engineer, you will be the backbone of our Martech data ecosystem. You will design, build, and optimize the data pipelines that power real-time promotions and personalized marketing journeys. We need someone who can take a business requirement from a whiteboard and turn it into a production-ready, end-to-end deliverable within our Databricks and AWS S3 environment.

Key Responsibilities:

Data Lakehouse Architecture:
Design and implement robust Data Lake patterns on AWS S3 using Databricks/Delta Lake to ensure data is scalable, searchable, and secure.
ETL/ELT Development:
Build end-to-end data pipelines to ingest, transform, and load data from various Martech sources, ensuring high quality and low latency.
Streaming Integration:
Collaborate on the integration of Apache Flink event streams into the data lake to support real-time promotional analytics and historical auditing.
Marketing Automation Support:
Facilitate data flows between the Data Lake and Braze, ensuring our marketing automation tool has the "golden record" of user data for personalized campaigns.
Performance Tuning:
Optimize Spark jobs and SQL queries to manage large-scale datasets efficiently, keeping an eye on both performance and AWS cost management.
Autonomy & Delivery:
Own the full development lifecycle—from requirement gathering and data modeling to deployment and monitoring—without needing constant hand-holding.

Technical Requirements:

Data Platforms: 5+ years of hands-on experience with Databricks (Spark, Delta Lake, Unity Catalog).
Cloud

Infrastructure: Deep expertise in AWS S3, including bucket policies, storage classes, and data organization best practices.
Streaming Knowledge:
Familiarity with Apache Flink or similar stream processing frameworks (e.g., Spark Streaming, Kafka Streams) for event-driven architectures.

Languages:

Expert-level proficiency in Python (PySpark) and SQL.
Martech

Experience:

Proven experience working with marketing data or tools like Braze, Salesforce Marketing Cloud, or similar CDPs/automation platforms.
Data Modeling:
Strong understanding of Medallion Architecture (Bronze/Silver/Gold) and Schema-on-read principles.

Soft Skills:

Independent Problem Solver:
You don’t wait for a ticket to tell you what’s broken; you find the bottleneck and fix it.
Clear Communicator:
You can explain complex data flows to non-technical stakeholders in the marketing team.
Delivery-Focused:
You have a "get it done" attitude with a high bar for code quality and documentation.

Preferred Qualifications:

Experience with Infrastructure as Code (Terraform/Cloud Formation).
Knowledge of CI/CD pipelines for data engineering (Git Hub Actions, Jenkins, or Azure Dev Ops).
AWS Certified Data Engineer or Databricks Certified Professional.