Data Engineer

Contract position with a major investment firm. Hybrid 3 days/week in Newark, NJ.

• Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experience
• 8+ years of experience as Data Engineer on AWS Stack with experience on Dev Ops tool
• AWS Solutions Architect or AWS Developer Certification required
• Solid experience of AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, EMR/Spark, RDS, Redshift, Data Sync, DMS, DynamoDB, Lambda, Step Functions, IAM, KMS, SM, Event Bridge, EC2, SQS, SNS, Lake Formation, Cloud Watch, Cloud Trail
• Implement high velocity streaming solutions and orchestration using Amazon Kinesis, AWS Managed Airflow and AWS Managed Kafka (preferred)
• Solid experience building solutions on AWS data lake/data warehouse
• Analyze, design, development, and implementation of data ingestion pipeline in AWS
• Knowledge implementing ETL/ELT for data solutions
• End-to-end data solutions (ingest, storage, integration, processing, access) on AWS
• Knowledge implementing RBAC strategy/solutions using AWS IAM and Redshift RBAC model
• Build & implement CI/CD pipelines for EDP Platform using Cloud Formation and Jenkins
• Programming experience with Python, Shell scripting and SQL
• Knowledge of analyzing data using SQL Stored procedures
• Build automated data pipelines to ingest data from relational database systems, file systems, NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift
• Build automated data pipelines to ingest data from Rest APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift.
• Good Experience of Dev Ops practice
• Experience with modern source code management and software repository systems (Bitbucket)
• Experience with programming languages (Python)
• Experience with scripting languages (Shell, Groovy)
• Experience with API deployment for tooling integration
• Experience using Jenkins, Cloud Bees (Pipeline as Code, Shared Libraries)
• The ability to document exceptions/issues/action plans/meeting minutes/lessons learned accurately and in a timely fashion
• Experience with administering Dev Ops tools in SaaS
• Experience using Dev Ops Tools (Sonar Qube, Artifactory etc.)
• Experience using build tools (Maven, MS Build and Gradle)
• Experience using containers (Docker)
• Experience using Atlassian suite (Jira, Confluence)
• Experience of Infrastructure as Code using Cloud Formation
• Creating Jenkins CI pipelines to integrate Sonar/Security scans and test automation scripts
• Part of Dev Ops QA and AWS team focusing on building CI/CD pipeline
• Responsible for writing and maintaining Jenkins Pipelines

• Designing, building, and maintaining efficient, reusable, and reliable code
• Ensure the best possible performance and quality of high scale data applications and services
• Participate in system design discussions
• Independently perform hands on development and unit testing of the applications
• Collaborate with the development team and build individual components into enterprise data platform
• Work in a team environment with product, QE/QA, and cross functional teams to deliver a project throughout the whole software development cycle.
• Responsible to identify and resolve any performance issues
• Keep up to date with new technology development and implementation
• Participate in code review to make sure standards and best practices are met
• Project management:
  Agile developers take responsibility for estimating, planning, and managing all tasks and report on progress
• Software quality:
  The Agile developer is also responsible for the quality of the software he/she produces. The team takes responsibility for the quality of the work they produce instead of turning over some code to a separate and independent group for testing
• Teamwork:
  This includes collaboration with all other team members with the aim to take shared responsibility for the overall efforts
• Understanding user needs:
  This is about interacting with users as necessary to clarify requirements