Sr. Databricks Data Engineer, Vice President, Hybrid

Who We Are Looking For

We are seeking a Senior Databricks Data Engineer with extensive technical, people, and process skillsets who can contribute to the development, enhancement, and maintenance of data platforms as part of an agile scrum team.

• Design, build, and test end‑to‑end data pipelines, including streaming, events, and batch ingestion on Azure‑based infrastructure.
• Develop and implement ETL processes for data ingestion, transformation, and loading into data lakes.
• Work extensively on Databricks, Delta Lake, and Apache Spark to build custom, high‑throughput, configurable frameworks and libraries.
• Drive change through collaboration, influence, and demonstration of POCs.
• Own all aspects of the software development lifecycle: design, coding, integration testing, deployment, and documentation.
• Collaborate within an agile project team and adhere to best practices and coding standards.

• 10+ years of overall Big Data pipeline experience, with 5+ years in Databricks and 5+ years in cloud‑based development (Azure services, Azure Dev Ops).
• Strong experience in micro‑services architecture, data warehousing platforms (Databricks, Delta Lake, Apache Spark), and scalable data processing.
• Hands‑on development in Databricks SQL and Scala; deep understanding of clusters, jobs, and resource management.
• Proficiency in monitoring, troubleshooting, and optimizing data pipelines.
• Data quality implementation and validation to ensure accuracy.
• Experience with Dev Ops pipelines (ADO, Harness), version control (Git), shell scripting, and database tools (SQL, TOAD).
• Excellent critical thinking, communication, and problem‑solving skills.

• Bachelor’s degree in Computer Science, Information Technology, or related field.
• Experience in the financial industry and agile development methodology.
• Knowledge of semi‑structured data formats (Avro, JSON, XML) and Docker, Kubernetes.
• Experience with Snowflake is a plus.

• Effective written and oral communication in a professional manner.
• Team player with a positive attitude, enthusiasm, initiative, and self‑motivation.
• Ability to multi‑task, energetic, fast learner, and problem solver.
• Will work 4 onsite days a week in office and one remote day; business hours 8am to 5pm.

Salary Range: $120,000 – $202,500 per year. The range quoted applies to the role in the specified location.

• 401K retirement savings plan with company match.
• Basic life, medical, dental, vision, long‑term disability and optional additional coverages.
• Paid time off: vacation, sick leave, short‑term disability, and family care responsibilities.
• Employee Assistance Program.
• Incentive compensation, including eligibility for annual performance‑based awards.
• Tax‑advantaged savings plans.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment.