Senior Azure Data Migration & Dataverse Integration Engineer Job Baltimore area,Maryland USA,Software Development

Company:
Arch Systems

Location:

Remote
Type:
Full-time
Position Summary

Seeking a Senior Azure Data Migration & Dataverse Integration Engineer to design, develop, implement, and support enterprise data migration and synchronization capabilities between Microsoft SQL Server and Microsoft Dataverse
.

The position requires a senior hands-on engineer capable of owning the complete data pipeline lifecycle, including source analysis, data mapping, extraction, transformation, migration, synchronization, validation, exception handling, monitoring, performance optimization, and automated deployment.

The engineer will develop and maintain repeatable pipelines supporting initial bulk migration, incremental migration, Change Data Capture (CDC), ongoing synchronization, and transition/cutover activities between legacy SQL Server applications and modern Microsoft Power Platform/Dataverse solutions.

Primary Responsibilities
The Data Migration Engineer will:

Design and implement end-to-end SQL Server-to-Dataverse data migration pipelines
.
Develop automated pipelines using Azure Data Factory (ADF) and supporting Azure services.
Analyze legacy SQL Server databases, tables, views, relationships, stored procedures, and data dependencies.
Map legacy SQL Server data structures to Dataverse tables, columns, relationships, choices, lookups, and alternate keys.
Change Data Capture
Scheduled synchronization
Dataverse-to-legacy synchronization where required
Cutover and final synchronization
Develop and maintain ADF pipelines, datasets, linked services, integration runtimes, triggers, parameters, variables, and reusable pipeline components.
Develop transformation and data-cleansing logic required to convert legacy data into Dataverse-compatible structures.
Implement controlled sequencing to maintain parent/child relationships and referential integrity.
Develop lookup resolution and cross-reference processes for Dataverse GUIDs and legacy SQL identifiers.
Implement restartable and recoverable migration processes.
Develop pipeline logging, audit, exception handling, retry, and reconciliation capabilities.
Validate migrated data for completeness, accuracy, integrity, and consistency.
Perform source-to-target record-count and field-level reconciliation.
Troubleshoot failed records, pipeline failures, API limitations, data-quality issues, and performance problems.
Develop automated deployment processes for data pipeline components across DEV, TEST, UAT, and PROD environments.

The candidate must have advanced hands-on experience designing and developing Azure Data Factory solutions.

Responsibilities include:

ADF pipeline architecture and development
Copy Activities
Lookup Activities
Stored Procedure Activities
For Each and Until processing
Conditional processing
Dynamic expressions
Linked Services
Datasets
Integration Runtime configuration
Managed Identity authentication
Pipeline triggers and scheduling
Incremental load processing
CDC patterns
Retry and failure handling
Pipeline logging and auditing
Performance tuning and parallelization
Environment-specific configuration
Infrastructure and pipeline deployment automation

The engineer should be capable of developing reusable, metadata-driven pipeline patterns rather than creating independent hard‑coded pipelines for every table.

SQL Server Responsibilities

Strong SQL Server development and data engineering skills are required, including:

Complex T-SQL development
Stored procedures
Views
Common Table Expressions
Temporary and staging tables
Data profiling
Data cleansing
Data transformation
Joins and relationship analysis
Primary and foreign key analysis
Indexing and query optimization
SQL Server Change Data Capture
Change Tracking
Watermark-based incremental extraction
Referential-integrity validation

The engineer must be capable…