Sr. Data Engineer
Listed on 2026-07-09
-
Software Development
Data Engineering, Python
Job Description
Sr. Data Engineer is needed to perform the following duties:
Build streamline pipelines leveraging various AWS services including Lambda and Glue for data processing, ingestion, transformation, feature engineering, and business analysis.
Leverage AWS Event Bridge, EMR and Step Functions to create a streamline data pipeline to invoke Lambda and Glue jobs with batch based and event-based triggers, finally ingesting transformed data into Snowflake.
Develop custom PySpark and Scala coding to perform data transformations such as filtering, cleansing, mapping, handling null values and improving data quality.
Improve efficiency by optimizing complex SQL queries in Snowflake, implementing data partitioning, and indexing strategies for parallel processing, and reducing pipeline complexity.
Leverage python and its multiple packages to create functions to create derived variables from existing variables, performed end-to-end testing, unit testing, integration tests, and exceeded the required sonar coverage.
Bachelor's Degree is required in Computer Science or Computer Engineering or Information Systems or Business Analytics.
#J-18808-Ljbffr(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).