Data Platform Lead; Hybrid

Overview

Data Platform Lead (Hybrid) – Chicago, IL

The American Medical Association (AMA) is the nation's largest professional association of physicians and a nonprofit organization. We are a unifying voice and powerful ally for America's physicians, the patients they care for, and the promise of a healthier nation. To be part of the AMA is to be part of our mission to promote the art and science of medicine and the betterment of public health.

At AMA, our mission to improve the health of the nation starts with our people. We foster an inclusive, people-first culture where every employee is empowered to perform at their best. Together, we advance meaningful change in health care and the communities we serve. We encourage and support professional development for our employees, and we are dedicated to social responsibility.

We have an opportunity at our corporate offices in Chicago for a Data Platform Lead (Hybrid) on our Information Technology team. This is a hybrid position reporting into our Chicago, IL office, requiring 3 days a week in the office.

• Serve as the platform owner for AMA's enterprise Databricks lakehouse, accountable for its technical direction, delivery, and operational health.
• Translate the enterprise data platform strategy and multi-year roadmap into an actionable technical delivery plan and engineering backlog.
• Define and maintain the platform's technical architecture, including workspace topology (development, staging, production), Delta Lake design, and Unity Catalog structure.
• Maintain a prioritized platform engineering backlog using agile delivery methods, balancing new capability development, technical debt remediation, and operational work.
• Evaluate emerging Databricks and data ecosystem capabilities and recommend adoption where they advance reliability, cost efficiency, or business value.
• Contribute to platform budget planning and forecasting, and provide technical input into platform-related procurement and vendor management.

• Owns platform prioritization and backlog, including intake and sequencing of requests from business units and engineering teams. Own the technical onboarding of new data sources and business-unit workloads onto the platform.
• Collaborate with Sr Data Engineer to lead the design, build, and operation of data ingestion, transformation, and delivery pipelines across batch and streaming workloads.
• Ensure adherence to engineering standards for data modeling, code quality, version control, testing, and CI/CD across the platform.
• Design and maintain self-service analytics capabilities and reusable data products that reduce duplicated effort across business units.
• Provide and operate shared MLOps platform capabilities (e.g., model registry, deployment frameworks), enabling data science and engineering teams to operationalize models.
• Develop and maintain platform documentation, runbooks, and standard operating procedures.
• Serve as the final decision authority on platform architecture and engineering standards; provide design approvals and guardrails for data engineering teams.

• Contribute to the reliability, performance, scalability, and security of the enterprise data platform, in partnership with the Director II and both IT and federated business-unit engineering teams.
• Define and operate observability tooling and monitoring practices for platform health, cost optimization, model performance, data quality, and governance compliance; drive continuous improvement initiatives.
• Drive cloud cost optimization through cluster configuration, capacity planning, storage management, and cost allocation across business units.
• Implement platform security controls, including encryption, access management, and audit logging, in alignment with IT Security policy and HIPAA and other regulatory requirements.
• Define, configure, and enforce platform-level governance controls (e.g., Unity Catalog, access policies, lineage tooling, data quality monitoring, validation, and issue-escalation processes).
• Partner with the Data Governance Lead to translate Enterprise Data Office policies into platform configuration and automated enforcement.
• Implement technical controls supporting AI governance, including model lifecycle management, monitoring, and drift detection.

May include other responsibilities as assigned.