Principal AI​/Machine Learning Data Engineer - Remote or hybrid from MN or DC

Position: Principal AI /Machine Learning Data Engineer - Remote or hybrid from MN or DC
Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by diversity and inclusion, talented peers, comprehensive benefits and career development opportunities.

Come make an impact on the communities we serve as you help us advance health equity on a global scale. Join us to start Caring. Connecting. Growing together.

The Enterprise Information Security (EIS) team is responsible for cybersecurity across our organization. We support our business and members by reducing risk, rapidly responding to threats, focusing on business resiliency and securing new acquisitions.

The Principal AI / Machine Learning Data Engineer role focuses on designing and building scalable data platforms that enable advanced analytics, machine learning, and AI-driven solutions. This role will support the development of intelligent systems that process large-scale event and operational data, enabling faster insights, automation, and decision-making across the organization.

This position sits at the intersection of data engineering, machine learning, and AI, with an emphasis on building modern data pipelines and enabling production-grade AI capabilities.

Ideal Candidate Profile:

* Demonstrated experience building and operating production data platforms and pipelines across batch and streaming workloads

* Solid hands-on engineering in Python and SQL; familiarity with JVM languages (Java/Scala) in Spark ecosystems is a plus

* Experience with distributed processing and lakehouse/warehouse patterns (eg, Spark/PySpark, Databricks, Snowflake)

* Experience building ingestion frameworks for structured and unstructured data, including event/log and semi-structured formats

* Experience enabling Generative AI solutions in production (eg, RAG-style architectures), including retrieval patterns and evaluation/monitoring practices

* Familiarity with knowledge-centric data approaches (eg, metadata-driven systems, entity resolution, and/or graph concepts) to improve discoverability and downstream analytics

* Solid data quality, observability, and monitoring mindset (profiling, validation, alerting, and reliability improvements)

* Comfort with orchestration, CI/CD, containerization, and infrastructure-as-code (eg, Airflow, Git Hub Actions, Docker, Terraform, Kubernetes)

* Cloud experience (AWS, Azure, and/or GCP), including secure handling of sensitive data (PII/PHI) and collaboration with compliance partners

* Ability to lead through influence, mentor engineers, and translate ambiguous problems into scalable technical roadmaps

You'll enjoy the flexibility to work remotely
* from anywhere within the U.S. as you take on some tough challenges. For all hires in the Minneapolis or Washington, D.C. area, you will be required to work in the office a minimum of four days per week.

Primary Responsibilities:

* Design, develop, and maintain scalable data pipelines and data platforms supporting analytics, machine learning, and AI use cases

* Build and optimize ingestion frameworks for large-scale structured and unstructured data, including streaming and event-driven sources

* Partner with cross-functional stakeholders to understand evolving data and AI needs and define long-term technical solutions

* Enable and support machine learning and AI workflows, including feature engineering, data preparation, and model deployment support

* Drive strategic initiatives around Generative AI, data quality, observability, lineage, and governance

* Develop and maintain frameworks that support rapid experimentation and deployment of AI/ML solutions

* Introduce and evolve best practices in data modeling, orchestration, testing, and monitoring

* Identify and champion opportunities for platform scalability, performance optimization, and cost efficiency

* Collaborate with product, analytics, and infrastructure teams to deliver high-impact data and AI solutions

* Build and maintain reusable parsing, enrichment, analytic, and service libraries to accelerate delivery across teams

* Work comfortably under time-sensitive conditions while ensuring thoroughness

* Maintain high ethical standards and the ability to remain objective and confidential

You'll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in.

Required Qualifications:

* Bachelor's degree or equivalent experience

* 5+ years of experience designing, building, and operating production data pipelines and platforms

* 5+ years of hands-on development with Python (preferred) and/or Java, including code reviews, packaging, and deployment

* 5+ years of experience with Spark (PySpark) and Databricks (or similar distributed…