Senior Data Engineer

This role is an integral senior member of a technical team, responsible for supporting the design, development, and maintenance of the organization's data infrastructure, ensuring the efficient and reliable flow of data across various systems. This position is responsible for ensuring the reliable movement, transformation, and storage of data across various platforms. The incumbent will work closely with data professionals, analysts, and engineers to build and optimize data pipelines, ensuring data quality and integrity.

The ideal candidate is adaptable, solution-oriented, and capable of collaborating across multiple teams to support a broad range of data initiatives. The incumbent and team will be involved in every aspect of the data process, from idea generation, analysis, design, build, and support, using the latest technologies and design patterns.

Highmark Inc.

This role is an integral senior member of a technical team, responsible for supporting the design, development, and maintenance of the organization's data infrastructure, ensuring the efficient and reliable flow of data across various systems. This position is responsible for ensuring the reliable movement, transformation, and storage of data across various platforms. The incumbent will work closely with data professionals, analysts, and engineers to build and optimize data pipelines, ensuring data quality and integrity.

The ideal candidate is adaptable, solution-oriented, and capable of collaborating across multiple teams to support a broad range of data initiatives. The incumbent and team will be involved in every aspect of the data process, from idea generation, analysis, design, build, and support, using the latest technologies and design patterns.

• Design, develop, and maintain robust data processes and solutions to ensure the efficient movement and transformation of data across multiple systems
• Develop and maintain data models, databases, and data warehouses to support business intelligence and analytics needs
• Collaborate with stakeholders across IT, product, analytics, and business teams to gather requirements and provide data solutions that meet organizational needs
• Monitor work against production schedule, provide progress updates, and report any issues or technical difficulties to lead developers regularly
• Implement and manage data governance practices, ensuring data quality, integrity, and compliance with relevant regulations.
• Collaborate on the design and implementation of data security measures, including access controls, encryption, and data masking
• Mentor other associate and intermediate data engineers as needed
• Perform data analysis and provide insights to support decision-making across various departments
• Stay current with industry trends and emerging technologies in data engineering, recommending new tools and best practices as needed
• Other duties as assigned or requested.

• 5 years of experience in design and analysis of algorithms, data structures, and design patterns in the building and deploying of scalable, highly available systems
• 5 years of experience in a data engineering, ETL development, or data management role.
• 5 years of experience in SQL and experience with database technologies (e.g., MySQL, PostgreSQL, MongoDB).
• 5 years of experience in data warehousing concepts and experience with data warehouse solutions (e.g., Snowflake, Redshift, Big Query)

• 5 years of experience with Python and Spark framework
• 2 years of experience with Databricks7 years of experience with data pipeline and workflow management tools (e.g., Apache Airflow, AWS Glue, Azure Data Factory).
• 7 years of experience defining system architectures and exploring…