AWS Python Developer Pyspark Newark, NJ

Job Summary

We are seeking an experienced Python developer with strong expertise in AWS and PySpark to join our data engineering team. The ideal candidate will have hands-on experience developing scalable data pipelines, processing large data sets, and integrating with cloud-based environments. This role requires excellent problem-solving skills and a strong understanding of distributed data processing frameworks.

• Design, develop, and maintain data pipelines and ETL workflows using Python, PySpark, and AWS services.
• Build and optimize large-scale data processing and data transformation solutions.
• Integrate various data sources and ensure data quality, performance, and reliability.
• Collaborate with data engineers, analysts, and architects to deliver end-to-end data solutions.
• Implement best practices for code optimization, error handling, and data validation.
• Participate in code reviews, documentation, and deployment automation.
• Ensure adherence to data security and compliance standards.

• Bachelor’s degree in Computer Science, Data Engineering, or a related field.
• 10+ years of experience in software development with a strong focus on Python.
• Hands-on experience with PySpark for distributed data processing.
• Solid understanding of AWS cloud services such as S3, Glue, Lambda, EMR, Redshift, and Athena.
• Strong experience in ETL development and data pipeline orchestration.
• Familiarity with SQL and relational/non-relational databases.
• Excellent analytical, debugging, and communication skills.

• Experience with Airflow, Databricks, or other workflow management tools.
• Knowledge of CI/CD pipelines and version control tools like Git.
• Exposure to data lake or data warehouse architectures.
• Familiarity with Docker or Kubernetes for deployment.