Senior Data Engineer

Direct message the job poster from Jarvis Consulting Group

Jarvis is driven by a bold vision to transform the professional services landscape by setting a new standard for innovation, empowerment, and impactful solutions in data, AI, and technology. Our unwavering dedication is rooted in propelling businesses toward sustained success while making a positive impact on communities. Strategically operating from Toronto and Montreal, Europe, and North Africa, Jarvis extends its services across Canada and the US.

Our diverse team comprises expert advisors and forward-thinking technologists who work tirelessly to unlock the full potential of businesses, ensuring the consistent delivery of exceptional results. The world of technology is rapidly evolving through innovation and cutting-edge technologies. As we move into the future, we believe in keeping up with relevant technologies and building that future ourselves. Our greatest assets are our people, and at Jarvis, we value imagination and curiosity as much as capability and skill.

We have an exciting opportunity for the right person - do you have Big Data experience, Spark ETL, Scala, Databricks, SQL development and want to be part of a new team? We are looking for you. You will be working with one of the best new consulting companies and have an integral role in their new data consulting practice.

• Understand Big Data and Cloud environments from a performance and capacity management standpoint, using the tools needed to onboard, monitor and move data into these environments
• Collaborate as the Big Data and Cloud Technical lead with multiple technology teams to ensure that the appropriate associated applications, integrations, infrastructure, and security architecture are designed to meet evolving business requirements, meet standards for reliability, scalability, performance, and availability
• Be the lead for internal and external products and projects. Helping to build world class big data software that solves real problems
• Responsible for the successful implementation of technical solutions for projects, supporting highly complex business applications with complex integration needs across multiple technology discipline
• Formulate and define project scope, and objectives based on a thorough understanding of the technical requirements of the projects
• Use sound Agile development practices (code reviews, unit testing, etc.) to develop and deliver quality code and data products
• Provide day-to-day support and technical expertise to both technical and non-technical teams
• Help to develop junior talent and transform them into experts
• Work with other engineers to brainstorm solutions to problems and support bank objectives
• Build out scalable and reliable ETL pipelines and processes to ingest and transform data from a large number and variety of data sources

• 5+ years Spark Development experience using Pyspark or related technologies such as Scala-Spark, Spark SQL; experience with Spark streaming highly preferred.
• Experience with Azure Services (ADLS, ADF, AKS etc)
• Experience with Databricks (required)
• Experience with streaming technologies such as Kafka and Spark Streaming
• Experience with Agile development (JIRA / Confluence)
• Experience with version control systems, such as Git
• Knowledge of dev tools like IDE’s, Maven, SBT
• Knowledge of orchestration tools like Airflow, Autosys, Oozie and Cron
• Experience with containerization (Docker) and orchestration (Kubernetes) is a plus
• Experience with developing and deploying applications to the cloud environment

• Works well both individually and as part of a team
• Proven ability to work creatively in a problem-solving environment
• Able to work closely with technical and non-technical team in a collaborative environment
• University degree in relevant STEM discipline (Computer Sciences, Electrical/Computer/Software Engineering and Mathematics)

Jarvis Consulting Group is committed to developing a barrier-free recruitment process and work environment. We are committed to equality in the workplace and to growing a welcoming and inclusive team. We encourage applications from Indigenous persons, visible minority group members, and people of all sexual orientations and genders, and will consider all qualified applicants without regard to race, colour, religion, gender, gender identity or expression, sexual orientation, national origin, or disability or age.

Jarvis is committed to compliance with all fair employment practices regarding citizenship and immigration status.

Accommodations are available on request for candidates taking part in any stage of the selection process. If you are a job seeker who requires assistance, please contact: