Senior Data Engineer; Airflow Specialist

Position: Senior Data Engineer (Airflow Specialist)
Job title:

Senior Data Engineer (Airflow Specialist)
Company:

Hvantage Tech Solutions Pvt. Ltd.
Department:

Enterprise Applications / Healthcare IT

Location:

Remote / Hybrid / Onsite (as applicable)

Employment Type:

Full-Time
Experience

Required:

5+ Years
Work Mode:

Must be comfortable working with global teams and US clients
Methodology:

SAFe Agile (Mandatory)

About Hvantage Tech Solutions Pvt. Ltd
.Hvantage Tech Solutions Pvt. Ltd. is a global technology and healthcare IT services provider delivering advanced digital solutions for healthcare payers, providers, and life sciences organizations. Our expertise spans enterprise platforms, healthcare interoperability, AI-enabled analytics, cloud-native architectures, and digital member engagement platforms

.
Role Overvi
ewWe are seeking a versatile Senior Data Engineer to lead the design, implementation, and scaling of our enterprise data orchestration platform using Apache Airflow. This role requires a "builder" mindset, combining the agility to create robust pipelines from the ground up with the maturity to ensure they are scalable, secure, and integrated into a complex cloud ecosystem. As the primary architect for workflow automation, you will move beyond simple task scheduling to build a resilient, self-healing data infrastructur

e.

Key Responsibili
ties Orchestration & Pipeline Engineer ing:
Author complex, modular, and idempotent DAGs using Python and the Airflow Task Flow
API.Framework Developm ent:
Build custom Operators, Hooks, and Sensors to standardize data integrations across the organizat
ion.

High-Compute Integrat ion:
Integrate Airflow with heavy-compute environments including Azure Databricks, Spark clusters, and cloud data warehou
ses.

Platform Managem ent:
Optimize Airflow executors (Celery or Kubernetes) to manage high-concurrency workloads and handle backend database tuning (PostgreSQL/MyS
QL).Security & Complia nce:
Implement enterprise-grade security protocols, including Role-Based Access Control (RBAC), Secret Management (Azure Key Vault/Hashi Corp Vault), and OAuth integrat
ion.

Dev Ops & Reliabil ity:
Build and maintain automated CI/CD deployment pipelines for DAGs and infrastructure-as-code using Terraform or Bi
cep.

Observabil ity:
Implement advanced monitoring and alerting using Prometheus, Grafana, or the ELK stack to track pipeline health and SLA breac
hes.

Leadership & Mentors hip:
Establish data engineering best practices across the team and conduct rigorous code reviews to ensure architectural integr

ity.

Technical Requir
ements

Orchestr ation:
Expert knowledge of Apache Airflow 3.x+ (specifically XComs, Providers, and Dynamic Task Map
ping).Lang uages:
Advanced proficiency in Python and expert-leve
l SQL.Cloud Plat forms:
Proficiency in Azure (Data Factory, Blob Storage, AKS) or equivalent services in AW
S/GCP.D evOps:
Hands-on experience with CI/CD tools such as Git Hub Actions, Azure Dev Ops, or Je

nkins.

Qualif
ications

Exp erience: 5+ years in Data Engineering, with a minimum of 2-3 years focused specifically on scaling Airflow in production envir
onments.

Ed ucation:
Bachelor’s or Master’s degree in Computer Science, Information Systems, or a relate
d field.

Preferred

Skills:

Experience migrating legacy schedulers to
Airflow.

Background in software development (Java o
r .NET).Knowledge of data quality frameworks like Great Expectations

or dbt.

Co
mpetencies

Strong focus on long-term maintenance and architectural
integrity.

Ability to move fast in an agile environment without compromising on cod
e quality.

Exceptional problem-solving skills for debugging complex data orchestrati

on issues.

Applicat i
on Process
-Interested candidates may apply at:  or Whats App at +9