Lead Data Engineer

Lead Data Engineer
Notice period
- Should be immediate only
Experience
-7 to 25 years
Location
- Hyderabad only
Work mode
- All 5 days' work from office

CORE RESPONSIBILITIES

• Hands-on delivery oversight: review pipelines, SQL, ETL/ELT logic, Spark jobs, reporting outputs and technical deliverables.

• Mentor junior talent: coach data engineers, data analysts and data quality analysts through daily guidance, reviews and troubleshooting.

• Data quality and controls: define validation, reconciliation, anomaly checks, lineage awareness and issue triage for finance/reporting data.

• Engineering governance: enforce coding standards, documentation, reusable patterns, production-readiness and maintainable data solutions.

• Stakeholder communication: clearly communicate risks, blockers, dependencies and corrective actions to delivery and programme stakeholders.

• SQL: advanced joins, window functions, query tuning, data validation and troubleshooting.

• Python: scripting, data processing, API invocation, automation and pipeline support.

• ETL/ELT: ingestion, transformation, scheduling, monitoring, error handling and operational support.

• Apache Spark:
Spark SQL, Data Frames/Datasets, batch pipelines, structured-streaming concepts, partitioning and performance tuning basics.

• Scala / PySpark: ability to understand or develop Spark transformations using Scala or PySpark;
Scala hands-on preferred where Spark workloads are Scala-based.

• Data modelling and governance: warehousing basics, quality checks, reconciliation, auditability and traceable reporting outputs.