Principal Data Engineer

Principal Data Engineers at Mayo Clinic are expected to be subject matter experts in building ETL pipelines, working with heterogenous data using different technologies, and database design and optimizations. The data they work with are required to support a different analyses and applications that serve both internal and external partners. As the position requires strong data, technical, and functional skills, the right candidate must be able to understand business requirements, translate them into information needs, and implement those requirements using available data.

The hire will be responsible for creating new data architecture, optimizing existing databases, designing data collection and flows for cross functional teams. The ideal candidate is an data engineer who has extensive experience working with clinical data and collaborating across diverse teams to ensure the data provided is accurate, the data pipelines reliable, and the resulting databases are fit for a variety of purposes.

The Principal Data Engineer will work with informaticists, clinicians, software engineers, and data scientists to serve as a subject matter expert in data engineering best practices and will ensure optimal data delivery architecture is consistent throughout ongoing projects. The position will also be responsible for the management of data related projects and oversee the tasks of other team members when relevant.

Core job duties:

• Expert in data modeling concepts,
• Responsible for designing, implementing, documenting and testing databases containing clinical data.
• Leverage SQL, Python and other programming languages
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Build ETL pipelines from varying data sources and technologies using cloud technologies such as Google Cloud or Microsoft Azure and implemented with Apache Airflow or similar products
• Problem solving with the initiative to think critically to identify improvement opportunities.
• Provide direction on best practices for data transformation, structure, and workflows.
• Ability to effectively participate in multiple, concurrent projects
• Research new and existing data sources in order to contribute to new development, improve data management processes, and make recommendations for data quality initiatives.
• Perform periodic data quality reviews for internal and external data.