Senior Storage Analyst GOAPRDJP

Overview

The Data and Content Management Division uses a one-government approach to information and privacy governance, decision-making, and service delivery across the Government of Alberta (GoA) balanced with individual client needs. This approach enhances data access, fosters collaboration, reduces data duplication, and drives innovation, ensuring efficient and effective services that ultimately benefit Albertans.

As part of this mandate, the Data Centre of Excellence (DCoE) requires a resource to oversee the platform and cluster administration of a Hadoop-based integrated data and analytics platform/enterprise data lake environment (Cloudera Data Platform). This role will also involve leading or assisting in the development, administration, and support of other tools and platforms, including but not limited to Azure and Snowflake.

These efforts will support the implementation of the GoA Data Strategy, enabling the government to better leverage its data assets. Working within a team, this resource is an expert and plays a key role in ensuring that cloud and on-premises systems are designed, developed, operated, and maintained to meet the needs of ministries and external stakeholders. This role will be instrumental in realizing the pillars of the GoA Data Strategy by using data to deliver meaningful outcomes for Albertans.

The successful candidate will be a Storage Analyst / Platform Analyst / Platform Developer with a diverse range of technical skills and experience in working in multi-faceted roles. The role may have aspects of any or all the following:
Data platforms (on-premises Hadoop-based and cloud) management and day to day administration, cloud architecture design and integration, security and compliance, ITIL-based service management, Agile delivery and Dev Ops practices, data governance and metadata management and stakeholder engagement and user support.

• Perform system administration and support of tools and platforms, including but not limited to Cloudera Data Platform.
• Lead or assist with development, administration, and support of other tools and platforms, including but not limited to Azure and Snowflake.
• Configure and tune cloud-based tools to manage cost, security, and compliance for the environments. Working across a myriad of technology stacks in Cloudera, Azure, Snowflake etc.
• Provide services to support data privacy and security in adherence to enterprise policies (access control, security policies across platforms, etc.).
• Deploy sophisticated analytics programs, AI/machine learning, and statistical methods.
• Develop technical design, perform capacity arrangement, computational cluster set up, performance fine-tuning, monitoring, structure planning, scaling, and overall GOA data lake/ platform administration.
• Collaboration with other team members to improve and optimize Cloud and on-premises infrastructure systems including scalability, reliability, performance, capacity, availability and disaster recovery
• Work with infrastructure, network, database, and application teams to ensure data lake components are highly available and performing within agreed-upon service levels.
• Implement and/or maintain relevant components of the Hadoop ecosystem. Examples:
  Apache YARN, Map Reduce, HDFS, Hive, Zookeeper, Pig, Spark, Zeppelin, Atlas, Kerberos, Ranger, etc.
• Install, administer, and support Linux and Windows operating systems in an enterprise environment.
• Accountable for Hadoop platform and storage performance tuning and volume management of cluster and routines.
• Perform setup, configuration, and security for the Hadoop cluster including integration with Microsoft Active Directory (AD).
• Monitor Hadoop cluster connectivity and performance.
• Manage and analyze Hadoop log files, optimize Hadoop services allocation for memory and CPU cores as well as garbage collection optimization.
• HDFS support and maintenance. File system management and monitoring. Provide file system services (file structure, access, sharing, quotas and security rights).
• Develop and document best practices.
• Set up new Hadoop users through integration with AD User and Groups as well as help set up security policies through Apache Ranger.
• Manage and administer Hadoop infrastructure on Linux Red Hat Enterprise Linux (RHEL) 7.x and higher versions.
• Adhere to existing Government of Alberta established policies, procedures, standards, guidelines and practices.
• Maintain, support, and enhance enterprise storage software/hardware products.
• Provide support for backup and recovery services including the hardware and software required to manage a large enterprise environment.
• Maintain/update storage procedures, processes documentation, configuration diagrams, and spreadsheets.
• Establish proper communications with clients, vendors, and internal support groups.
• Provide required technical assistance and consultation to internal technical teams as well as other ministries, as needed.
• Establish and operate control processes including schedule…