Senior Data Engineer: Scalable Pipelines & Analytics

Overview

This position is responsible for aiding the organization in leveraging both internal and external data assets to achieve business insights, gain deeper understanding of member needs, and enhance tactical and strategic decision‑making. The Data Engineer position will support the architecture and systems that facilitate data movement within the organization in both Linux and Windows environments, including managing the software that runs in those environments.

Collaborates with data science teams, software developers, business analysts, and data engineers to extract transactional, historical, analytical, behavioral, and predictive data sets by building data pipelines from ever‑increasing data resources. Understands progressive statistical, programmatic, and analytical approaches to business hypotheses as they apply to available data, and aids business owners and the data science team in the interpretation of source data and the associated business data pipelines.

Requires a solid understanding of both the technical and business aspects of Data Science and business analytics, as well as full software‑development life‑cycle experience within various analytical architectures, including piloting models and processes, architecting underlying data design, and model/process operationalization.

Will be required to explore internal and external data sources to identify useful structures within the data that were or are previously unknown to the business. Should have a strong understanding of relational and dimensional modeling to facilitate data wrangling, model operationalization, and exploration of data. Responsible for adhering to corporate data‑governance policies and controls to ensure the accuracy, timeliness, and confidentiality of resources under management.

Must be able to work with users at all levels of management in developing and implementing operational and strategic analytics. The Data Engineer’s primary focus will be on operationalization of data pipelines to facilitate use of novel and optimized data‑set products by a variety of end users using a variety of data‑transformation tools.

• Manage installation, patching, upgrading, and administration of software systems that support data pipelines. This includes ETL platforms (Airflow, SSIS), data‑quality platforms (Great Expectations), distributed data‑processing systems (Hopsworks, Polar), and streaming platforms (Kafka, Rabbit
  
  MQ).
• Develop data pipelines to support the operational implementation of Data Science and analytical models.
• Ensure all security, documentation, governance, and compliance standards are met.
• Oversee the development of a small reporting layer on top of analytical models to ensure accountability and insight into models’ performance and impact.
• Perform Dev Ops and orchestration around supporting data marts and analytical processes.
• Mentor Data Engineer I, II, III levels to train them on best practices and optimal technology usage.
• Work with management regarding forecasting, planning, and implementation of used and future technologies leveraged to support data engineering processes.

• Engineer each solution with appropriate kinds of technology, with optimized performance.
• Execute analytics solution engineering strategy, requirements, and adapt processes as needed.
• Monitor and secure usage of data while tracking platform product performance.
• Advise and collaborate in internal partnerships to deliver accurate, available, and accessible API services.
• Identify potential concerns based on projections of current trends.
• Contribute to Credit Union culture by keeping a positive attitude and growing relationships.

EDUCATION:

• Bachelor of Science degree in information technology, computer science, statistics, or related business degree with equivalent experience of 10+ years.
• Master of Science in information technology, computer science, statistics, or related degree with equivalent experience of 5+ years.

EXPERIENCE:

• Minimum of 7 years experience in designing, developing, and implementing data solutions.

QUALIFICATIONS:

• Must have advanced SQL abilities, experience with both Oracle and Microsoft…