Big Data Developer

Job Description

CPS Data CoE - Big Data Engineer

• This position is for Big data engineer for Client Wealth Management Framework CoE team at client’s Alpharetta or New York offices.
• CoE team is responsible to define and govern the data platforms.
• We are looking for colleagues with strong sense of ownership and ability to drive solutions.
• The role is primarily responsible to automate the existing process and bring new ideas and innovation.
• The candidate is expected to code, conduct code reviews, and test framework as needed, along with participating in application architecture and design and other phases of the automation.
• The ideal candidate will be a self‑motivated team player committed to delivering on time and should be able to work with or without minimal supervision.

Design & Develop new automation framework for ETL processing

• Support existing framework and become technical point of contact for all related teams
• Performance tuning of spark, snowflake ETL jobs
• New technology POC and suitability analysis for Cloud migration
• Process optimization with the help of automation and new utility development
• Work in collaboration for any issues and new features
• Support any batch issue
• Support application team teams with any queries

• 7 plus years of Data engineering experience
• Must be strong in UNIX Shell, Python scripting knowledge
• Must be strong in Spark
• Must have strong knowledge of SQL
• Hands‑on knowledge on how HDFS/Hive/Impala/Spark works
• Strong in logical reasoning capabilities
• Should have working knowledge of Github, Dev Ops, CICD / Enterprise code management tools
• Strong collaboration and communication skills
• Must possess strong team‑player skills and should have excellent written and verbal communication skills
• Ability to create and maintain a positive environment of shared success
• Ability to execute and prioritize a task and resolve issues without aid from direct manager or project sponsor
• Good to have working experience on snowflake & any data integration tool i.e. informatica cloud

Python, Big Data & Apache Spark

Snowflake/Azure/AWS any cloud

• IDMC/any ETL tool

Mid‑Senior level

Contract

Information Technology

IT Services and IT Consulting