ETL Test Automation Engineer – Azure Data Platform

The QA & Automation Engineer will be responsible for validating ETL pipelines, data migration processes, and cloud data platform integrations across Azure services. This role involves hands‑on testing of large-scale data platforms, automation development, advanced SQL validation, and ensuring end‑to‑end data quality across enterprise data engineering initiatives.

Key Responsibilities

Perform end‑to‑end testing of ETL pipelines built using Azure Data Factory, Azure Databricks, Azure Synapse, and SSIS.

Validate data transformations, mappings, data quality rules, and data lineage.

Conduct source‑to‑target (S2T) reconciliation, data profiling, completeness, and accuracy checks.

Verify schema changes, incremental loads, delta loads, and historical load processes.

Data Migration Testing

Design and execute test strategies for large‑scale on‑prem to cloud or cross‑cloud data migration projects.

Validate ETL/ELT processes and post‑migration data accuracy.

Perform count checks, checksum validation, CDC validation, duplicate checks, and referential integrity validation.

Review and validate data mapping documents, business rules, and acceptance criteria.

Data Lake & Cloud Platform Testing

Test ingestion pipelines into Azure Data Lake Storage (ADLS Gen2) from multiple source systems.

Validate partitioning, folder structures, file formats (Parquet, CSV, JSON), and governance standards.

Conduct performance and scalability testing for large data workloads.

Validate integration flows across Data Lake, Synapse, ADF, and Power BI.

Automation Testing

Develop automated data validation and regression frameworks using Python, PySpark, and ADF automated validation frameworks.

Integrate automated tests within CI/CD pipelines using Azure Dev Ops.

Build reusable automation templates, accelerators, and validation utilities.

Create and maintain test plans, scenarios, and test cases in Azure Dev Ops or Jira.

Log, track, triage, and validate defects in collaboration with development and data engineering teams.

Publish daily/weekly QA health metrics, defect reports, coverage summaries, and quality dashboards.

Experience and Qualifications

4–10 years of experience in Data Warehouse / Data Engineering QA.

Strong hands‑on expertise with ETL tools such as ADF, SSIS, Informatica, and Databricks.

Expert‑level SQL for complex data validation and reconciliation.

Strong understanding of data warehousing concepts (SCDs, facts/dimensions, star schema).

Strong analytical, debugging, and problem‑solving skills.

Preferred Skills

Quality management and defect lifecycle management.

Integration of automated tests into CI/CD pipelines using Azure Dev Ops.

#J-18808-Ljbffr