Data Migration & Batch Engineer
Listed on 2026-07-19
-
IT/Tech
Data Engineering, Data Analyst
Data Migration & Batch Engineer
Locations are Austin, Charlotte, San Diego, NYC-Onsite
ResponsibilitiesOwn schema-drift detection and reconciliation (Red Gate / SQL comparison) — the deterministic ground truth on which every downstream decision rests.
Build the deterministic rule ladder for table and job dispositions: staging / truncate-and-reload detection, dead-table detection, target-equivalent matching, straight-through repoint, duplicate-job merge.
Own the
-collision engine: a full (never sampled) overlap scan across every shared ; classification into true collision, phantom and historical overlap; analysis of the resolution strategy; and a non-bypassable validation harness on every script that touches an n.
Own data reconciliation: row counts, column checksums, referential integrity, stored-procedure output parity, business-key parity; staging promotion gates and validated rollback scripts.
Batch: build the dependency graph, blast-radius (butterfly) scoring, Simple/Medium/Complex classification, and the migration approach per job.
Own the SLA validation harness — proving every migrated job and every SLA chain runs within its window under combined nightly volume.
Emit the downstream artifacts: migration scripts, job modifications, rollback scripts, reconciliation specifications.
QualificationsSQL Server (2024) — deep: DDL, stored procedures, indexing, referential integrity, query performance.
Relational data migration at scale — Red Gate, AWS DMS or equivalent; cutover, reconciliation and rollback delivered in production.
Identity and key-collision resolution — surrogate versus natural keys, reconciliation and historical-mapping tables, preserving referential integrity under a merge. Rare, and highly valued for this role.
Informatica Power Center; ETL dependency analysis; SLA and batch-window modelling.
Python and strong SQL; data-quality and validation frameworks.
The discipline to keep correctness-critical steps deterministic while working alongside an LLM reasoning layer.
Working knowledgeInformatica IDMC and/or Apache Airflow;
PySpark.
Graph queries; handling of regulated / PII data.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).