Python + PySpark - Junior Developer
Listed on 2025-12-02
-
IT/Tech
Data Engineer, Cloud Computing, Big Data, Data Science Manager
The Python + PySpark + AWS Consultant will be responsible for designing, developing, and optimizing data engineering solutions using Python and PySpark. The role requires hands-on experience in building scalable ETL pipelines, working with cloud platforms such as AWS or Azure, and ensuring data integrity and performance optimization. The candidate should have strong SQL skills and a deep understanding of data engineering principles.
Key Responsibilities
• Develop and maintain data processing workflows using Python and Py Spark
• Design and implement ETL pipelines for structured and unstructured data
• Optimize Spark-based data processing for efficiency and scalability
• Deploy and manage data solutions on AWS or Azure
• Write and optimize SQL queries for data transformation and analysis
• Troubleshoot and resolve performance issues in data pipelines
• Work closely with cross-functional teams to ensure data reliability and integrity
Required Qualifications
• 5+ years of experience in data engineering
• Strong proficiency in Python and object-oriented programming
• Hands-on experience with PySpark for large-scale data processing
• Proficiency in SQL for data manipulation and query performance tuning
• Experience with AWS or Azure for cloud-based data solutions
• Knowledge of ETL processes and data pipeline automation
• Experience with Hadoop is acceptable
Preferred Qualifications
• Experience in optimizing Spark jobs for performance and cost efficiency
• Familiarity with Dev Ops practices for data engineering
• Understanding of data governance, security, and compliance best practices
#J-18808-Ljbffr(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).