Sr. Data Engineer

The Role

Fortellar delivers large scale data migration and integration programs for clients in regulated industries. This role builds the pipelines behind that work: extracting data from legacy systems of record, conforming it into a governed lakehouse, integrating it with modern enterprise business platforms, and extending the same foundation into reporting and analytics.

The data is regulated and frequently sensitive. Accuracy is not negotiable. Every migrated record must reconcile to source, and every transformation must be explainable to an auditor.

This is a hands‑on builder role. You will own pipelines, data modelling, business rules, reconciliation, and reporting.

• Build and operate change data capture and snapshot ingestion from relational systems of record, together with reference and lookup data retrieved through REST and OData interfaces.
• Design conformed, canonical data models that resolve fragmented source tables into a single trusted record per business entity.
• Implement incremental change detection and publishing with deduplication, checkpointing, and idempotent replay.
• Build and operate outbound integration to enterprise business platforms.
• Implement and document transformation logic that encodes business rules, with a named owner recorded for every mapping decision.
• Build reconciliation between source and target at record, control total, and business measure level, and report it as a standing metric.
• Implement data quality controls including constraint enforcement, quarantine of rejected records, and freshness monitoring.
• Promote work through source control, automated tests, and deployment tooling across development, test, and production, designing jobs that run unattended under service identities.
• Build reporting tables, semantic models, and analytics ready datasets on the delivered foundation.
• Advise client stakeholders directly, challenge assumptions with evidence, and produce documentation that clients rely on.

• 6+ years in data engineering, including at least three years building production pipelines.
• Deep practical command of Delta Lake, including MERGE behavior and cost, Change Data Feed, optimization, and schema evolution.
• Experience with change data capture ingestion from relational sources, including late, duplicate, and out of order events.
• Hands‑on experience with the Microsoft Azure data stack, including Azure Data Factory, Azure Data Lake Storage, and a Spark-based lakehouse compute platform.
• Advanced SQL and strong PySpark, writing tested, reviewable, source‑controlled code rather than notebook scripts.
• Experience delivering system to system integration with guaranteed delivery semantics, covering idempotency, deduplication, checkpointing, and dead letter handling.
• Experience with environment-promotion deployment tooling for lakehouse/Spark workloads (bundle- or CI/CD-based) and promotion across environments.
• Experience implementing data quality and reconciliation controls in an environment subject to external audit.
• Direct client stakeholder experience and excellent written communication.