Sr. Data Engineer; Hybrid

Sr. Data Engineer (Hybrid)

Chicago, IL

The American Medical Association (AMA) is the nation's largest professional Association of physicians and a non-profit organization. We are a unifying voice and powerful ally for America's physicians, the patients they care for, and the promise of a healthier nation. To be part of the AMA is to be part of our Mission to promote the art and science of medicine and the betterment of public health.

At AMA, our mission to improve the health of the nation starts with our people. We foster an inclusive, people-first culture where every employee is empowered to perform at their best. Together, we advance meaningful change in health care and the communities we serve.

We encourage and support professional development for our employees, and we are dedicated to social responsibility. We invite you to learn more about us and we look forward to getting to know you.

We have an opportunity at our corporate offices in Chicago for a Sr. Data Engineer (Hybrid) on our Information Technology team. This is a hybrid position reporting into our Chicago, IL office, requiring 3 days a week in the office.

As a Sr. Data Engineer, you will play a key role in implementing and maintaining AMA's enterprise data platform to support analytics, interoperability, and responsible AI adoption. This role partners closely with platform engineering, data governance, data science, IT security, and business stakeholders to deliver high‑quality, reliable, and secure data products. This role contributes to AMA's modern lakehouse architecture, optimizing data operations, and embedding governance and quality standards into engineering workflows.

This role serves as a senior technical contributor within the team-providing mentorship to junior engineers and implementing engineering best practices within the data platform function, in alignment with architectural direction set by leadership.

RESPONSIBILITIES:

Data Engineering & AI Enablement

* Build and maintain scalable data pipelines and ETL/ELT workflows supporting analytics, operational reporting, and AI/ML use cases.

* Implement best practice patterns for ingestion, transformation, modeling, and orchestration within a modern lakehouse environment (e.g., Databricks, Delta Lake, Azure Data Lake).

* Develop high‑performance data models and curated datasets with strong attention to quality, usability, and interoperability; create reusable engineering components and automation.

* Collaborate with the Architecture Team, the Data Platform Lead, and federated IT teams to optimize storage, compute, and architectural patterns for performance and cost‑efficiency.

* Build model-ready data sets and feature pipelines to support AI/ ML use cases; serve as a technical coordination point supporting business units' AI-related infrastructure needs.

* Collaborate with data scientists and AI Working Group to operationalize models responsibly and maintain ongoing monitoring signals.

Governance, Quality & Compliance

* Embed data governance, metadata standards, lineage tracking, and quality controls directly into engineering workflows; ensure technical implementation and alignment within engineering workflows.

* Work with the Data Governance Lead and business stakeholders to operationalize stewardship, classification, validation, retention, and access standards.

* Implement privacy‑by‑design and security‑by‑design principles, ensuring compliance with internal policies and regulatory obligations.

* Maintain documentation for pipelines, datasets, and transformations to support transparency and audit requirements.

Platform Reliability, Observability & Optimization

* Monitor and troubleshoot pipeline failures, performance bottlenecks, data anomalies, and platform‑level issues.

* Implement observability tooling, alerts, logging, and dashboards to ensure end‑to‑end reliability.

* Support cost governance by optimizing compute resources, refining job schedules, and advising on efficient architecture.

* Collaborate with the Data Platform Lead on scaling, configuration management, CI/CD pipelines, and environment management.

* Collaborate with business units to understand data needs, translate them…