Lead Software Engineer - Databricks​/PySpark​/AWS

Lead Software Engineer - Databricks / PySpark / AWS

Posted 3 days ago — Be among the first 25 applicants.

$/yr - $/yr

We have an exciting and rewarding opportunity for you to take your software engineering career to the next level. As a Lead Software Engineer at JPMorgan Chase within Corporate Technology, you play a crucial role in an agile team dedicated to enhancing, building, and delivering trusted, market-leading technology products in a secure, stable, and scalable manner. As a key technical contributor, you are tasked with implementing critical technology solutions across multiple technical domains, supporting various business functions to achieve the firm’s business objectives.

• Design solutions at the appropriate level of detail and drive consensus among peers as needed.
• Champion software engineering best practices within the team.
• Collaborate with software engineers and cross-functional teams to design and implement deployment strategies using AWS Cloud and Databricks pipelines.
• Lead the design, development, testing, and implementation of application solutions.
• Partner with technical experts, stakeholders, and team members to resolve complex technical challenges.
• Proactively address issues to support leadership objectives and prevent customer impact.
• Design, develop, and maintain robust data pipelines for ingesting, processing and storing large volumes of data from diverse sources.
• Implement ETL (Extract, Transform, Load) processes to ensure data quality and integrity using tools such as Apache Spark and PySpark.
• Monitor and optimize the performance of data systems and pipelines.
• Apply best practices for data storage, retrieval and processing.
• Maintain comprehensive documentation for data systems, processes, and workflows.

• Formal training or certification on software engineering concepts and 5+ years applied experience.
• Proficiency in programming languages such as Python and PySpark.
• Experience designing and implementing data pipelines in cloud environments.
• Strong background in design, architecture and development using AWS Services, Databricks, Spark, Snowflake, and related technologies.
• Experience with CI/CD tools such as Jenkins, Git Lab, or Terraform.
• Familiarity with containerization and orchestration technologies including ECS, Kubernetes, and Docker.
• Ability to troubleshoot issues related to Big Data and Cloud technologies.

• 5+ years of experience leading and developing data solutions in AWS Cloud.
• 10+ years of experience building, implementing, and managing data pipelines using Databricks on Spark or similar cloud technologies.

We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity or expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law.

We also make reasonable accommodations for applicants’ and employees’ religious practices and beliefs, as well as mental health or physical disability needs. Visit our FAQs for more information about requesting an accommodation.

JPMorgan Chase & Co. is an Equal Opportunity Employer, including Disability/Veterans.