Senior Python Developer Spark

Senior Python Developer with Spark Position Description

Senior Python Developer with Spark - design, build, and optimize large-scale data processing systems in a cloud-native AWS environment. This role focuses heavily on developing high-performance data pipelines using Spark and Python, as well as working with complex relational datasets.

Located at a client site in Reston, VA with a hybrid working model.

Diagnose and improve inefficient data workflows, especially Spark jobs, and write advanced SQL to support hierarchical and analytical use cases. Partner with cross‑functional teams to deliver scalable, reliable, and well‑architected data solutions that support critical business functions in a financial services setting.

• 8+ years of hands‑on experience with Python for data engineering, including building and maintaining data pipelines
• Deep expertise in Apache Spark, including performance tuning (partitioning, caching, broadcast joins, shuffle optimization), understanding of execution plans (DAGs, stages, tasks), memory and resource management
• Solid experience with big data ecosystems such as Hadoop, Hive, and EMR
• Advanced proficiency in SQL, including writing recursive CTEs for hierarchical data, query optimization, indexing strategies, and execution plan analysis
• Strong experience with AWS services including EMR, Lambda, Step Functions, Event Bridge, Redshift, S3, and Glue
• Experience building and consuming APIs, along with data transformation and ingestion workflows
• Proven ability to work with large-scale datasets, performing data analysis and extracting actionable insights
• Familiarity with data modeling concepts (normalized/denormalized structures, handling hierarchical data)
• Hands‑on experience with CI/CD pipelines and tools such as Git Lab and Terraform
• Strong understanding of performance troubleshooting, including identifying bottlenecks in distributed systems
• Ability to clearly explain technical decisions, especially around Spark optimization and SQL logic
• Strong analytical thinking, problem‑solving skills, and attention to detail
• Effective collaboration skills in cross‑functional, matrixed environments

• Experience in financial services or regulated environments
• Exposure to data visualization tools
• Familiarity with event‑driven architectures on AWS

Bachelor's degree in Computer Science, Information Systems, or a related field.

Compensation range: $99,200.00 - $.

• Competitive compensation
• Comprehensive insurance options
• Matching contributions through the 401(k) plan and the share purchase plan
• Paid time off for vacation, holidays, and sick time
• Paid parental leave
• Learning opportunities and tuition assistance
• Wellness and Well‑being program

Qualified applicants will receive consideration for employment without regard to their race, ethnicity, ancestry, color, sex, religion, creed, age, national origin, citizenship status, disability, pregnancy, medical condition, military and veteran status, marital status, sexual orientation or perceived sexual orientation, gender, gender identity, and gender expression, familial status or responsibilities, reproductive health decisions, political affiliation, genetic information, height, weight, or any other legally protected status or characteristics to the extent required by applicable federal, state, and/or local laws where we do business.

CGI provides reasonable accommodations to qualified individuals with disabilities. If you need an accommodation to apply for a job in the U.S., please email the CGI U.S. Employment Compliance mailbox at U You will need to reference the Position  the position in which you are interested.

All CGI offers of employment in the U.S. are contingent upon the successful completion of a background investigation. Investigation components may vary depending on specific assignment and/or level of U.S. government security clearance held. Some investigations may include a credit check. CGI will consider qualified applicants with arrests and conviction records in accordance with all local regulations and ordinances.

CGI will not disclose pay information to individuals who do not otherwise have access to compensation information, except as permitted by law or formal investigation. Employees with access to compensation may disclose pay of other employees only in response to a formal complaint or charge, in furtherance of an investigation, proceeding, hearing, or action, or consistent with CGI’s legal duty to furnish information.