Senior Data Engineer – PySpark & Python (Client

Senior Data Engineer – PySpark & Python (Client:)

Location:

Charlotte, NC

Need Senior profiles with at least 12 years of experience.

• Design, develop, and optimise large-scale data pipelines using PySpark and Python.
• Implement and adhere to best practices in object-oriented programming to build reusable, maintainable code.
• Write advanced SQL queries for data extraction, transformation, and loading (ETL).
• Collaborate closely with data scientists, analysts, and stakeholders to gather requirements and translate them into technical solutions.
• Troubleshoot data-related issues and resolve them in a timely and accurate manner.
• Leverage AWS cloud services (e.g., S3, EMR, Lambda, Glue) to build and manage cloud-native data workflows (preferred).
• Participate in code reviews, data quality checks, and performance tuning of data jobs.

• Relevant experience in a data engineering or backend development role.
• Experience working with AWS cloud ecosystem (S3, Glue, EMR, Redshift, Lambda, etc.). Kafka
• Strong hands‑on experience with PySpark and Python, especially in designing and implementing scalable data transformations.
• Solid understanding of Object-Oriented Programming (OOP) principles and design patterns.
• Proficient in SQL, with the ability to write complex queries and optimise performance.
• Strong problem-solving skills and the ability to troubleshoot complex data issues independently.
• Excellent communication and collaboration skills.
• Hands‑on experience with AI Tools.

• Senior Data Engineer
• Py Spark
• Python
• Large-Scale Data Pipelines
• Data Pipeline Development
• Data Pipeline Optimization
• Object-Oriented Programming (OOP)
• Reusable Code
• Maintainable Code
• Advanced SQL
• SQL Queries
• Data Extraction
• Data Transformation
• ETL (Extract, Transform, Load)
• Data Loading
• Data Engineering Best Practices
• Data Workflow Development
• Cloud‑Native Data Workflows
• AWS
• Amazon S3
• Amazon EMR
• AWS Lambda
• AWS Glue
• Data Processing
• Big Data Technologies
• Data Integration
• Data Quality Checks
• Performance Tuning
• Data Job Optimization
• Code Reviews
• Troubleshooting
• Data Issue Resolution
• Collaboration with Data Scientists
• Collaboration with Analysts
• Stakeholder Requirements Gathering
• Technical Solution Design