Kafka Data Engineer

Location: Montreal

Job Description

Experience 2-5 years

Business Background:
Position is for a role in TEDRA department. TEDRA is part of the Institutional Securities Technology (IST) Division. It is responsible for maintaining, distributing, and revenue, risk, and reference data (client, product, and pricing). As the authoritative source of key data sets, we are at the forefront of database technology and are heavily involved in data engineering, data science, data visualization, and machine learning efforts across the Firm.

Position

Introduction:
This is a Kafka data engineer role in the team responsible for developing on the firm’s Capture data stores that holds the transactional bigdata for real time and archive processing and getting it into the archives and data lake. The global team consists of highly technical team members who are adaptable to both hands-on development and project management. We deliver multiple projects for multiple business areas in parallel.

The business owners and subject matter experts will be globally distributed, making communication and pro-active to be important. You will be expected to work closely with our operations partners on requirements for projects.

The development will be performed using an agile methodology which is based on scrum (time boxing, daily scrum meetings, retrospectives, etc.) and XP (continuous integration, refactoring, unit testing, etc) best practices. Candidates must therefore be able to work collaboratively, demonstrate good ownership and be able to work well in teams.

Responsibilities:
– Configure and setup Kafka plants to support applications processing
– Build application specific monitoring and observability tools to help predict and prevent outages
– Collaborate with application and infrastructure teams to investigate production issues and provide solutions
– Provide support coverage during early Asia hours when needed Technical Skills Requirement:

* Practiced knowledge of Kafka

* Strong with scripting languages such as:
Python and Unix / K-Shell

* Knowledge of database concepts is desirable

* Familiar with lifecycle of data in an investment banking operation

* Experienced in Agile development process