Junior Architect​/Senior Data Engineer - Remote

Senior Azure Data Engineer

Optum Insight is improving the flow of health data and information to create a more connected system. We remove friction and drive alignment between care providers and payers, and ultimately consumers. Our deep expertise in the industry and innovative technology empower us to help organizations reduce costs while improving risk management, quality and revenue growth. Ready to help us deliver results that improve lives?

Join us to start Caring. Connecting. Growing together.

We are seeking a Senior Azure Data Engineer to help design, build, and operate our next-generation enterprise data platform on Microsoft Azure. You will own end-to-end delivery of data pipelines and data products that power analytics, regulatory reporting, operational dashboards, and emerging AI/ML use cases. You will partner closely with data architects, analytics engineers, data scientists, business stakeholders, and platform engineering teams to deliver reliable, performance, secure, and cost-efficient data solutions.

This role is ideal for an engineer with solid hands-on depth in Azure Data Factory, Azure Synapse Analytics and/or Databricks, and modern Lakehouse patterns, who is comfortable leading migration programs (e.g., Informatica-to-ADF, on-prem warehouse-to-cloud), mentoring mid-level engineers, and shaping engineering standards across the team.

You'll enjoy the flexibility to work remotely from anywhere within the U.S. as you take on some tough challenges.

For all hires in the Minneapolis or Washington, D.C. area, you will be required to work in the office a minimum of four days per week.

Primary Responsibilities:

• Pipeline Design & Development
  • Design and build robust, reusable, parameter-driven ingestion and transformation pipelines using Azure Data Factory, Synapse Pipelines, Data Bricks and/or Microsoft Fabric Data Factory
  • Implement medallion architecture (Bronze / Silver / Gold) on Azure Data Lake Storage Gen2 using Delta Lake, Parquet, and structured streaming patterns
  • Build performant ELT workflows that leverage pushdown to source systems (Synapse Dedicated SQL Pool, Azure SQL, Teradata) where appropriate
  • Develop and optimize PySpark notebooks and jobs on Azure Databricks or Synapse Spark
• Data Modeling & Warehousing
  • Design dimensional models (Kimball star/snowflake) and data vault patterns for analytics consumption
  • Implement Slowly Changing Dimensions (Type 1/2/3), Change Data Capture, and late-arriving data patterns
  • Tune distributed SQL workloads in Synapse Dedicated SQL Pool / Fabric Warehouse, including distribution keys, partitioning, and clustered column store indexes
• Platform Engineering & Dev Ops
  • Implement CI/CD for data pipelines using Azure Dev Ops (YAML pipelines, ARM/Bicep/Terraform) across Dev / SIT / UAT / Prod environments
  • Instrument pipelines with robust logging, auditing, and monitoring using Azure Monitor, Log Analytics, and KQL
  • Define and enforce coding standards, code review practices, branching strategies, and release management
• Migration & Modernization
  • Lead or contribute to legacy-to-cloud migrations - e.g., Informatica Power Center to Azure Data Factory, on-premises Teradata / Oracle / SQL Server to Synapse or Fabric
  • Perform workload assessment, capacity planning, and cost modeling for target-state architectures
  • Production incident response for critical pipelines

You'll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in.

Required Qualifications:

• Demonstrated experience in Data Bricks and Py Spark
• Production experience with one or more of:
  Azure Synapse Analytics (Dedicated and Serverless SQL Pools, Spark Pools) OR Azure Databricks (Delta Lake, Unity Catalog) OR Microsoft Fabric (Warehouse, Lakehouse, One Lake)
• Experience with Azure Key Vault, Azure AD / Entra  (including managed identities and service principals), and private networking (VNet integration, private endpoints)
• Experience with monitoring and troubleshooting with Azure Monitor, Log Analytics, and KQL
• Deep hands-on expertise with Azure Data Factory: pipelines, datasets, linked services, triggers, parameterization, mapping data flows, and all three Integration Runtime types (Azure, Self-hosted, SSIS)
• Demonstrated working knowledge of Azure Data Lake Storage Gen2 (hierarchical namespace, RBAC + ACLs, lifecycle management, security)
• Proficient in T-SQL; familiarity with Spark SQL, KQL, Power Shell, and Bash shell scripting
• Advanced SQL - window functions, CTEs, query optimization, execution plan analysis, performance tuning
• Demonstrated Python for data engineering - pandas, PySpark, REST API integration, unit testing (pytest)
• Willing/able to travel approximately 10% of the time as business needs dictate (occasional travel to Springfield, IL two to three times per year)

Preferred Qualifications:

• 5+ years of data warehouse development experience
• 5+ years of data modeling…