Senior Data Engineer

Overview

Are you passionate about transforming legacy data systems and driving enterprise-scale platform modernizations? Do you thrive in modern cloud environments and want to help shape our strategic migration to Microsoft Fabric? Join our innovative Data & Analytics team as a Senior Data Engineer
, where you'll play a pivotal role in refactoring, optimizing, and scaling our core enterprise data platform.

You’ll build robust, scalable PySpark pipelines and modern Lakehouse structures within our Azure Synapse environment, while also helping lay the foundation for our upcoming Microsoft Fabric migration in collaboration with external architecture partners and mentoring junior team members. This is your opportunity to make a lasting impact at the intersection of hands‑on data engineering and next‑generation modernization—solving complex pipeline bottlenecks today while architecting the ecosystem of tomorrow.

We embrace modern development tools and encourage the use of AI companions (like Git Hub Copilot or Google Gemini) to speed up workflows. However, we believe that true senior‑level engineering requires the foundational depth to question, validate, and ruthlessly scrutinize AI‑generated solutions before they ever hit production. Code accountability ultimately rests with the engineer.

This role will be located onsite in Camden, NJ.

• Lead the technical execution of refactoring legacy logic (predominantly SQL Stored Procedures) into modern, optimized PySpark pipelines within a Lakehouse architecture.
• Partner alongside a third‑party architecture vendor to help influence, design, and implement foundational frameworks, CI/CD pipelines, and governance standards for our upcoming Microsoft Fabric environment.
• Actively design, build, and optimize robust ETL/ELT pipelines, ensuring high performance, scalability, and data integrity across the Azure ecosystem.
• Take ownership of data platform health by refactoring fragile processes, minimizing technical debt, and building highly stable frameworks that reduce production bottlenecks.
• Act as a technical guide and escalation point for associate data engineers, establishing engineering best practices and setting up the team to successfully manage ongoing platform maintenance.
• Implement frameworks that ensure high data quality, strict security standards, and comprehensive lineage tracking across all data ingestion and migration paths.

• Experience:
  
  5+ years of dedicated data engineering or related experience, with a proven track record of designing, building, and deploying pipelines in enterprise cloud environments.
• SQL Mastery:
  Deep expertise in advanced T‑SQL, including extensive experience writing, debugging, and reverse‑engineering complex Stored Procedures.
• Advanced PySpark/Python:
  Strong proficiency in Python and PySpark for data processing, with the capacity to translate complex, legacy SQL‑driven logic into efficient notebook‑based pipelines.
• Azure Expertise:
  Strong hands‑on experience with core cloud services, specifically:
  • Azure Synapse Analytics
  • Azure Pipeline
  • Azure Data Lake Storage (ADLS Gen2)
• Critical Code Evaluation:
  Possess the deep, foundational understanding of SQL and PySpark internals required to critically evaluate, debug, and optimize complex code—whether written by peers or generated by AI systems.
• ETL/ELT Pipeline Management:
  Proven track record of building, running, and troubleshooting robust data pipelines in high‑availability production environments.
• Problem Solver: A passion for performance tuning, troubleshooting bottlenecked processes, and re‑engineering legacy code for maximum resilience.

• Conceptual understanding of, or early hands‑on exposure to, Microsoft Fabric tools (One Lake, Lake houses, Warehouses).
• Experience collaborating with external systems integrators, consultants, or architecture vendors to deploy enterprise‑level frameworks.
• Strong foundation in Git‑based workflows, Dev Ops principles, and CI/CD concepts in a data engineering ecosystem.

Base salary range: $ –  USD per annum. Position also eligible for an annual discretionary bonus, targeted at 3%.…