Senior Data Engineer Job Jersey City area,New Jersey USA,Software Development

Optomi, is seeking a Software Engineer to join a high-impact Data Engineering team responsible for building scalable, cloud-native data solutions. This role will focus on designing and developing data ingestion pipelines, ETL frameworks, and large-scale data processing applications utilizing AWS services such as Glue, EMR, S3, Lambda, and Redshift.
The ideal candidate will have strong hands‑on experience with AWS Glue and PySpark, a solid foundation in software engineering best practices, and experience building production‑grade data pipelines that move data from on‑premise environments into AWS cloud platforms. This individual will collaborate closely with engineering, architecture, and business teams to deliver secure, scalable, and resilient solutions in a fast‑paced Agile environment.

Responsibilities

Design, develop, test, and support scalable data engineering solutions using AWS Glue, PySpark, and EMR.
Build and maintain data ingestion pipelines from on‑premise systems into AWS cloud environments.
Develop ETL/ELT workflows leveraging AWS services including S3, Lambda, SQS, Event Bridge, Glue Studio, Glue Workflows, and Glue Data Catalog.
Create high‑quality, production‑ready code following software engineering best practices and SDLC standards.
Optimize Spark and Glue workloads for performance, scalability, reliability, and cost efficiency.
Develop and maintain data models, data warehouse solutions, and data processing frameworks.
Write complex SQL queries, stored procedures, and data transformation logic across relational and cloud‑based databases.
Collaborate with cross‑functional teams to translate business requirements into scalable technical solutions.
Participate in architecture discussions, code reviews, testing, deployment activities, and production support.
Implement monitoring, logging, and operational best practices to ensure application resiliency and stability.
Contribute to CI/CD pipelines and Infrastructure as Code initiatives within AWS cloud environments.
Support Agile delivery practices including sprint planning, backlog refinement, and continuous improvement initiatives.

Required Qualifications:

3+ years of professional experience in software engineering, data engineering, or ETL development.
Strong hands‑on experience with AWS Glue and PySpark in production environments.
Experience building and supporting ETL pipelines using AWS Glue Studio, Glue Workflows, and Glue Data Catalog.
Experience with AWS EMR and distributed data processing frameworks.
Proficiency with SQL and relational database technologies such as Oracle, Postgre

SQL, MySQL, or Aurora.
Experience designing and implementing data ingestion pipelines utilizing AWS services including:
S3
Lambda
SQS
Event Bridge
Cloud Watch
Knowledge of AWS security services including IAM and KMS.
Understanding of software development lifecycle (SDLC), testing methodologies, and production deployment processes.
Experience working within Agile/Scrum environments.
Strong troubleshooting, analytical, and problem‑solving skills.
Excellent communication and collaboration abilities.

Nice‑to‑Have

Qualifications:

Experience with Kafka, AWS MSK, Confluent Kafka, or other streaming technologies.
Experience with Terraform, Cloud Formation, or other Infrastructure as Code tools.
Experience with Redshift, Snowflake, Databricks, or modern cloud data warehouse platforms.
Knowledge of CI/CD pipelines and Dev Ops best practices in AWS environments.
Experience with Quick Sight, Sigma, Knowledge Graphs, or Graph

DB technologies.
Financial services, banking, or highly regulated industry experience.
AWS certifications (Solutions Architect, Data Engineer, Developer, or related certifications).
Experience supporting large‑scale enterprise data modernization or cloud migration initiatives.

#J-18808-Ljbffr