Databricks Engineer and Architect

Alternate Locations:
Radnor, PA (Pennsylvania)

Work Arrangement:

Hybrid :
Employee will work 3 days a week in a Lincoln office

Relocation assistance: is not available for this opportunity.

Requisition #: 75826

The Role at a Glance

As a Risk Data Engineer and Architect, you will architect and lead the design and implementation of new technology solutions for the Risk Investments and Technology team within the Chief Risk Office (CRO). You will own technical roadmaps, champion SDLC best practices, and build production grade systems that power analytics, machine learning, and cross-functional risk management.

Working on a centralized cloud data platform (AWS, Databricks or Dataiku), you'll design scalable data pipelines, implement data governance frameworks using modern cataloging tools, and develop advanced solutions. This role is critical to modernizing how we manage market risk, credit risk, and operational risk data on the enterprise scale. Significant Databricks experience is required.

You'll partner closely with risk analysts, quantitative researchers, and technology leaders to migrate business-owned processes into robust, production-controlled platforms while maintaining operational excellence.

What you'll be doing

* Architect cloud-native data solutions using AWS services (S3, Redshift, Glue, EMR, Lambda, Athena) and Databricks (Unity Catalog, Workflows, MLflow) or similar technology to support risk analytics at scale.

* Build and optimize data pipelines using Python, PySpark, and SQL to integrate structured and unstructured data from internal and external sources.

* Drive modernization initiatives by migrating legacy on-premises systems to cloud platforms while ensuring zero disruption to critical business services.

* Implement data governance frameworks including cataloging, lineage tracking, quality monitoring, and access controls aligned with internal controls and regulatory requirements.

* Collaborate cross-functionally with data scientists, risk managers, and business stakeholders to translate requirements into scalable technical solutions, drive change management and process improvements.

* Develop production-grade code following Dev Ops practices including CI/CD, automated testing, code reviews, and infrastructure-as-code.

* Manage and optimize relational databases (MySQL, Aurora Postgre

SQL, Redshift) for performance, cost efficiency, and data integrity.

* Integrate structured and unstructured data from various internal and external sources.

* Mentor and influence technical standards and architectural decisions across the Risk, Investments Technology organization.

* Monitor and troubleshoot production systems, implementing observability practices and proactive performance tuning.

What we're looking for

* Bachelor's degree in Computer Science, Information Systems, Engineering or related field.

* 5+ years of experience in data engineering or similar role implementing with a variety of on-premises and cloud data management, integration, and analytical technologies.

* Production experience with relational databases such as MySQL, and SQL Server, including designing schemas, writing complex queries, optimizing performance, and ensuring data integrity for a variety of business applications.

* Strong experience with end-to-end data architecture implementing systems that use AWS services related to data storage and management, must have advanced proficiency in SQL, Python, YAML and Bash.

* Thorough understanding of the Software Development Life Cycle (SDLC), including Dev Ops practices, CI/CD processes, application resiliency, and security measures.

* Excellent communication skills with ability to translate technical concepts for business audiences.

* Data governance experience: catalog tools, data quality frameworks, lineage tracking, and compliance controls is a plus.

* AWS certifications such as SAA, Associate Developer, Data Analytics Specialty, or Databricks certification is a plus.

Final date to receive applications

Applications for this position will be accepted through May 15th, 2026 subject to earlier closure due to applicant volume.

What's it like to work here?

At Lincoln Financial, we love what we do. We make meaningful contributions each and every day to empower our customers to take charge of their lives. Working alongside dedicated and talented colleagues, we build fulfilling careers and stronger communities through a company that values our unique perspectives, insights and contributions and invests in programs that empower each of us to take charge of our own future.

What's in it for you:

* Clearly defined career tracks and job levels, along with associated behaviors for each of Lincoln's core values and leadership attributes

* Leadership development and virtual training opportunities

* PTO/parental leave

* Competitive 401K and employee benefits

* Free financial counseling, health coaching and employee assistance program

* Tuition assistance program

* Work arrangements that work for you

* Effective productivity/technology…