Big Data Software Engineer

Overview

In this key role as a Software Engineer, you'll get to transform a Streamlit-built UI prototype into a polished, production-ready interface, with a roadmap to extend its reach and build it into an interface with future plans to make the interface available external to the corporate enterprise.

We don’t love the bullet points, but we love the work!

• Using the Linux CLI and Linux tools
• Developing Bash scripts to automate manual processes
• Recent software development experience using Python and Java
• Familiar with Streamlit, a Python framework for building interactive web applications
• Experience developing front-end applications using Type Script, HTML, and CSS
• Experience with JavaScript frameworks such as React, Angular, and Vue
• Familiar with Distributed Big Data processing engines including Apache Spark
• Experience using Jupyter Notebook
• Experience with data wrangling and preprocessing using tools such as pandas, Num Py
• Experience working with structured, semi-structured, and unstructured data such as Parquet, JSON, CSV, XML
• Familiarity with data quality concepts, data validation, and anomaly detection
• Experience with Git Source Control System

• Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows
• Experience with SQL technologies such as MySQL, Maria
  
  DB, and Postgre
  
  SQL for querying, joining, and aggregating large datasets
• Familiar with HPC Job Scheduling tools including Slurm
• Experience using the Atlassian Tool Suite (JIRA, Confluence)
• Appreciates a sense of humor and the occasional well-timed joke 😄