Senior Data Platform Engineer

About Aperia Solutions

Aperia Solutions is a leading payments SaaS company headquartered in Dallas, Texas. We build risk management, residuals accounting, compliance, and analytics platforms used by merchant banks, ISOs, payment processors, and other participants in the credit card payments ecosystem. Operating under PCI DSS 4.0 compliance across dual on-premises datacenters (Allen, TX and Chicago, IL) plus Azure cloud, we process high-volume payment data for some of the industry’s largest players.

We’re in the middle of a significant platform modernization program: a next-generation risk analysis and scoring platform on Microsoft Fabric, an AI-powered ingestion platform, new AI-native applications, and ongoing evolution of the products our clients rely on every day. The teams executing this work are deliberately small and senior, with executive-level support and minimal bureaucracy. Every senior hire is a founding contributor to the architecture and the code.

Aperia is building a next-generation risk analysis and scoring platform on Microsoft Fabric, and this role owns its medallion data architecture end-to-end. You'll design and build Bronze / Silver / Gold Delta tables, Fabric Spark feature pipelines, Direct Lake semantic models, and physical-layout optimizations. This is our highest-utilization engineering role on the project - central to both Phase 1 platform foundation and Phase 2 risk product delivery, working alongside a Senior Technical Lead, Senior Dev Ops engineer, and mid-level developer.

The platform ingests transaction and merchant data from clients via SFTP and API, tokenizes sensitive PAN data at the ingestion boundary using our reserved-BIN HMAC-derived format-preserving scheme, and lands data in Aperia's canonical Silver on Microsoft Fabric per our internal canonical spec. You'll build the pipeline that populates canonical Silver via MERGE, consume it via One Lake shortcuts, and materialize risk-specific Gold tables that feed both custom-authored DMN rules and pre-scored Mastercard Brighterion feeds.

• Design and implement Bronze, canonical Silver, and Gold Delta table schemas with PAN-safe physical layout — statistics allowlist configuration, liquid clustering, deliberate retention settings, V-Order optimization.

• Build Fabric Spark notebooks for the feature pipeline (rolling velocity features across 24h / 7d / 30d windows, merchant-level rolling features, hierarchical rollups, cross-merchant features via  joins).

• Author per-flavor-version Silver adapters — one function per flavor, unioned by name, tested against golden fixtures.

• Build the canonical MERGE DAG (Bronze → canonical Silver, idempotent on _natural_key per Aperia's canonical Fabric internal spec v1.4).

• Build the Gold materialization pipeline for both grains (_risk, _risk) with correct score-source stamping and idempotent merge semantics.

• Design and implement Direct Lake semantic models per reporting surface; performance-test against representative data volumes.

• Configure One Lake namespace layouts, workspace catalogs, and cross-workspace shortcut relationships with the residuals programme.

• Establish idempotent Delta write patterns using delta-rs from Python containers (segmented parallel writes with single atomic commit).

• Fabric capacity sizing (F-SKU selection, load testing, Reservation vs PAYG decisions).

• Partner with the Dev Ops engineer on Fabric IaC (Terraform capacity provisioning plus Fabric REST workspace bootstrap).

• Consult on Click House contingency if Direct Lake latency proves insufficient for client-tier UIs.

• 5+ years data engineering experience, with at least 2 years hands-on Spark development (PySpark preferred).

• Deep Delta Lake internals knowledge — transaction…