Principal Data Engineer – ML Platforms

Altarum Institute Principal Data Engineer ML Platforms Altarum | Data & AI Center of Excellence (CoE)

Altarum is building the future of data and AI infrastructure for public health - and were looking for a Principal Data Engineer ML Platforms to help lead the way. In this cornerstone role, you will design, build, and operationalize the modern data and ML platform capabilities that power analytics, evaluation, AI modeling, and interoperability across all Altarum divisions.

What You'll Work On

• ML Platform Engineering: modern lakehouse architecture, pipelines, MLOps lifecycle
• Applied ML enablement: risk scoring, forecasting, Medicaid analytics
• NLP/Generative AI support: RAG, vectorization, health communications
• Causal ML operationalization: evaluation modeling workflows
• Responsible/Trusted AI engineering: model cards, fairness, compliance

Your work ensures that Altarums public health and Medicaid programs run on secure, scalable, reusable, and explainable data and AI infrastructure.

What You'll Do

• Platform Architecture & Delivery
• Design and operate modern, cloud-agnostic lakehouse architecture using object storage, SQL/ELT engines, and dbt.
• Build CI/CD pipelines for data, dbt, and model delivery (Git Hub Actions, Git Lab, Azure Dev Ops).
• Implement MLOps systems: MLflow (or equivalent), feature stores, model registry, drift detection, automated testing.
• Engineer solutions in AWS and AWS Gov Cloud today, with portability to Azure Gov or GCP.
• Use Infrastructure-as-Code (Terraform, Cloud Formation, Bicep) to automate secure deployments.

Pipelines & Interoperability

• Build scalable ingestion and normalization pipelines for healthcare and public health datasets, including:
• FHIR R4 / US Core (strongly preferred)
• HL7 v2 (strongly preferred)
• Medicaid/Medicare claims & encounters (strongly preferred)
• SDOH & geospatial data (preferred)
• Survey, mixed-methods, and qualitative data

• Create reusable connectors, dbt packages, and data contracts for cross-division use.
• Publish clean, conformed, metrics-ready tables for Analytics Engineering and BI teams.
• Support Population Health in turning evaluation and statistical models into pipelines.

Data Quality, Reliability & Cost Management

• Define SLOs and alerting; instrument lineage & metadata; ensure 95% of data tests pass.
• Perform performance and cost tuning (partitioning, storage tiers, autoscaling) with guardrails and dashboards.

Applied ML Enablement

• Build production-grade pipelines for risk prediction, forecasting, cost/utilization models, and burden estimation.
• Develop ML-ready feature engineering workflows and support time-series/outbreak detection models.
• Integrate ML assets into standardized deployment workflows.

Generative AI Enablement

• Build ingestion and vectorization pipelines for surveys, interviews, and unstructured text.
• Support RAG systems for synthesis, evaluation, and public health guidance.
• Enable Palladian Partners with secure, controlled-generation environments.

Causal ML & Evaluation Engineering

• Translate R/Stata/SAS evaluation code into reusable pipelines.
• Build templates for causal inference workflows (DID, AIPW, CEM, synthetic controls).
• Support operationalization of ARAs applied research methods at scale.

Responsible AI, Security & Compliance

• Implement Model Card Protocol (MCP) and fairness/explainability tooling (SHAP, LIME).
• Ensure compliance with HIPAA, 42 CFR Part 2, IRB/DUA constraints, and NIST AI RMF standards.
• Enforce privacy-by-design: tokenization, encryption, least-privilege IAM, and VPC isolation.

Reuse, Shared-Services, and Enablement

• Develop runbooks, architecture diagrams, repo templates, and accelerator code.
• Pair with data scientists, analysts, and SMEs to build organizational capability.
• Provide technical guidance for proposals and client engagements.

Your First 90 Days – You will make a meaningful impact fast. Expected outcomes include:

• Platform skeleton operational: repo templates, CI/CD, dbt project, MLflow registry, tests.
• Two pipelines in production (e.g., FHIR analytics and claims normalization).
• One end-to-end CoE lighthouse MVP delivered (ingestion model metrics BI).
• Completed playbooks for Gov Cloud deployment, identity/secrets, rollback, and cost control.

Success…