Databricks Data Engineer

National Data Solutions | Hybrid — Franklin, TN | $65k-$80K

NDS is supporting a data warehouse and RCM modernization program for a large national healthcare client. The immediate need is a contract Databricks Data Engineer with strong healthcare claims data experience to build the pipeline and ingestion foundation and validate claims data integrity during the program's initial phase.

Build and configure nightly automated data extraction pipelines from multiple source systems including practice management, clearinghouse, and EHR platforms. Normalize and canonicalize claims data across heterogeneous sources into a unified enterprise data model. Validate claims data integrity, completeness, and accuracy — including charge, payment, adjustment, denial, and ERA/EOB data — against a 24-month historical baseline. Identify and document data quality gaps, feed failures, and reconciliation discrepancies.

Implement data lineage, refresh cadence, and governance controls within the Databricks environment. Support Power BI connectivity from the Databricks semantic layer. Coordinate directly with NDS domain leads and client stakeholders on data dictionary alignment, KPI logic, and validation findings. Participate in UAT and resolve data quality issues surfaced during dashboard validation.

3+ years of hands‑on Databricks engineering experience including Delta Lake, Unity Catalog, and Databricks Workflows. Direct experience with healthcare claims data — charge entry, claim submission, remittance, denial management, or AR — in a data engineering or analytics context. Demonstrated ability to validate and reconcile complex multi‑source claims datasets. Proficiency in PySpark and SQL. Experience connecting Databricks to Power BI via Direct Query or semantic layer.

Working knowledge of HIPAA data handling requirements and PHI masking in non‑production environments. Strong communication and client‑facing skills — you will interact regularly with NDS leads and client stakeholders and are expected to present findings, ask the right questions, and manage expectations professionally. Ability to work independently against a defined scope and surface blockers clearly and promptly.

Familiarity with clearinghouse data structures, ERA/EOB formats, or CARC/RARC code sets. Prior work in RCM, home health, or managed care analytics environments.