Software Engineer III - Senior Databricks​/Spark​/AWS Data Engineer

Build secure, scalable data engineering solutions in a role focused on modernizing to a Databricks-on-AWS lakehouse
. At JPMorgan

Chase, you’ll design and deliver production-grade pipelines using Apache Spark (PySpark), optimize for performance and cost, and help enable trusted analytics for business partners. The team also emphasizes a culture of diversity, opportunity, inclusion, and respect
, with support for engineering growth through communities of practice and emerging technologies.

• Design, build, and maintain new data pipelines on Databricks using Py Spark , delivering secure, high-quality production code and collaborating through reviewing and debugging work by others.
• Optimize PySpark jobs and Databricks clusters for performance
  , scalability
  , and cost efficiency
  , applying techniques such as partitioning
  , caching
  , and resource management
  .
• Create scalable data frameworks for end-to-end Databricks pipelines for workforce data analytics using medallion (bronze/silver/gold) lakehouse patterns.
• Implement data quality checks and validation using Delta Lake and Delta Live Tables expectations to support accurate, reliable data.
• Build monitoring and alerting to proactively address data ingestion issues
  , leveraging Databricks and AWS Cloud Watch to improve performance and throughput.
• Find opportunities to eliminate or automate remediation for recurring issues using Databricks Workflows and AWS-native automation
  .
• Use AI and Agentic AI to accelerate pipeline development, including adopting AI-assisted engineering tools such as Claude and Git Hub Copilot
  .
• Provision and deliver curated, reliable datasets for BI partners using Sigma, Tableau, and Alteryx to support reporting and analytics.
• Partner with stakeholders to understand requirements and design solutions, producing architecture and design artifacts for complex applications.
• Contribute to software engineering communities of practice that explore new and emerging technologies
  , reinforcing a strong culture of inclusion and respect.

• 3+ years of applied experience in data engineering, with formal training or certification in software engineering concepts.
• Advanced, hands‑on expertise in Apache Spark (PySpark) for large‑scale distributed processing, plus strong experience building and operating production pipelines on Databricks with Delta Lake and lakehouse patterns.
• Strong knowledge of the AWS data ecosystem
  , including S3, EMR, Glue, Lambda, and Athena
  , along with AWS storage and compute services; experience with Parquet and Iceberg formats.
• Strong programming skills in Python (Java or Scala is a plus).
• Experience with automation and continuous delivery
  , using CI/CD pipelines and tools such as Git/Bitbucket
  , Jenkins
  , or Spinnaker
  .
• Hands‑on experience spanning system design
  , application development
  , testing
  , and operational stability
  , including agile methodologies, resiliency, and security.
• In‑depth knowledge of the financial services industry and its IT systems.
• Solid SQL and data modeling skills, including experience with relational databases such as Oracle (a plus).
• Practical experience with scheduling tools such as Airflow and Autosys
  .

Databricks
, Apache Spark
, Py Spark ,
AWS
, Delta Lake
, Delta Live Tables
, Databricks Workflows
, AWS Cloud Watch
, AI
, Agentic AI
, Claude
, Git Hub Copilot
, Sigma
, Tableau
, Alteryx
, Python
, Java
, Scala
, Git
, Bitbucket
, Jenkins
, Spinnaker
, S3
, EMR
, Glue
, Lambda
, Athena
, Parquet
, Iceberg
, SQL
, Oracle
, Airflow
, Autosys
.

• Competitive total rewards package including base salary determined based on role, experience, skill set, and location.
• In eligible roles,
  commission‑based pay and/or discretionary incentive compensation paid as cash and/or forfeitable equity.
• Comprehens…