Sr. Databricks Data Engineer, Onsite, AVP

Overview

We are looking for a Sr. Databricks Data Engineer with hands‑on development experience on Databricks to process high volume data in near real‑time for State Street Global Technology Services.

• Design & develop custom high‑throughput data pipelines using Databricks.
• Design, build, and test end‑to‑end data ingestion pipelines (streaming, events, and batch), data integration, and data curation.
• Design, develop, and deploy scalable ETL pipelines on cloud‑based infrastructure using Databricks SQL & Scala/PySpark.
• Optimize and tune Databricks job performance, including query optimization, resource management, and scaling strategies.
• Address data issues and environment issues, performing impact analysis, root‑cause analysis, corrective and preventive maintenance.
• Responsible for all aspects of the software development lifecycle: design, coding, integration testing, deployment, and documentation.

• Bachelor’s Degree in a computer or IT related subject.
• 7+ years of overall IT experience.
• 5+ years of Data Engineering experience handling large datasets.
• 3+ years of Databricks hands‑on experience using SQL & Scala/PySpark.
• 3+ years of experience on cloud‑based development using Azure/AWS.
• Strong hands‑on skills in Databricks SQL & Scala/PySpark.
• Experience troubleshooting Dev Ops pipelines and Azure services.
• Expertise in end‑to‑end data solutioning for building a Data Lakehouse using Databricks.
• Experience with Micro Services – API/Event‑Driven Architecture/Development.
• Strong critical thinking, communication, and problem‑solving skills.
• Experience in agile development methodology.
• Experience working in the financial industry (preferred).
• Team player with a positive attitude, enthusiasm, initiative and self‑motivation.
• Ability to multi‑task, meet aggressive timelines, and maintain strong work ethics.

Onsite 4 days a week in office and one day remote. Standard business hours 8:00 AM to 5:00 PM.

Salary Range: $90,000 – $157,500 annual. Employees are eligible to participate in State Street’s comprehensive benefits program, which includes a 401(k) plan with company match, insurance coverage (basic life, medical, dental, vision, long‑term disability and optional additional coverages), paid time off (vacation, sick leave, short‑term disability, and family care), employee assistance program, incentive compensation (including eligibility for annual performance‑based awards), and tax‑advantaged savings plans.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.