Director, Data Engineer - PGIM Technology; Hybrid - Newark, NJ

Overview

Job Classification:
Technology – Engineering & Cloud.

As the Director, Data Lead for the PGIM Product Technology you will engage in requirements gathering, lead design, architect solutions and partner with business owners, technology leads, and engineering teams to support the shift to Microsoft cloud platforms for data processing and management. You will define and enforce standards while overseeing end‑to‑end data engineering—including sourcing, modeling, security, governance, and business intelligence functions—within the Microsoft Fabric platform.

The role is based in Newark, NJ and is hybrid; you are expected to be on site three days per week.

• Design and implement a configurable data ingestion ETL framework, including data quality, transformation, and modelling.
• Implement Lakehouse solutions using Microsoft Fabric and pave the road to PGIM’s future state using Azure services and Fabric infrastructure.
• Develop and deploy data integration solutions, handling ingestion, transformation, and delivery to consumers within Microsoft Fabric while using optimal storage formats and layers.
• Partner with Infrastructure/Dev Ops teams to build CI/CD pipelines that can be extended across the organization.
• Architect solutions that incorporate BI tools such as Power BI for seamless data integration, analytics, and reporting.
• Ensure data accuracy, consistency, and integrity across the data warehouse ecosystem through robust solution designs.
• Design and build AI solutions leveraging search, embeddings, retrieval, ranking, and Retrieval‑Augmented Generation (RAG).
• Set up a roadmap for AI‑as‑a‑service to consuming systems and business stakeholders.
• Establish a workspace architecture strategy by designing and implementing data warehouse and lakehouse solutions that allow data sharing between teams while avoiding duplication and adhering to security and governance rules.
• Lead a small team to design, build, and monitor data and AI solutions and ensure teams follow best practices for security, compliance, scalability, reliability, and performance.
• Serve as a subject‑matter expert in the Fabric space, providing hands‑on training to development teams and mentoring business areas to maximize Fabric AI capabilities.
• Create and maintain comprehensive documentation (playbooks) of solution architectures for internal and external stakeholders, covering Data Fabric architecture, processes, and procedures.
• Collaborate closely with business and technology partners—including data analysts, architects, engineers, and client stakeholders—to gather requirements and translate them into scalable Microsoft Fabric solutions.
• Proactively seek out new knowledge and adapt to emerging trends, technical solutions, and patterns that improve availability, reliability, efficiency, observability, and performance of products.
• Share knowledge with other engineers to drive consistency in monitoring and operations at scale.

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related discipline.
• 10+ years of experience in cloud computing with Microsoft Azure, leading migrations, integrations, and deployments.
• 10+ years as a Data Architect with demonstrated expertise in designing, managing, and deploying client solutions.
• Strong knowledge of Azure services and features (Synapse Analytics, Data Factory, Storage Accounts, Logic Apps, Functions, SQL Database, Active Directory, Dev Ops, Monitor, etc.).
• Deep familiarity with Fabric workloads and features (Notebooks, Deployment Pipelines, Workspace architecture and security, Data Warehouse/Lakehouse Architecture, Semantic Model, Data Factor, Data Flow, Data Activator, AI Skills, Copilot).
• Experience building intelligent applications with Azure AI Foundry, including prompt orchestration, grounding with Azure AI Search, and agent orchestration for enterprise‑scale GenAI use cases.
• Proficient in CI/CD pipelines using Git Hub Actions for deployment in Azure environments.
• Proven ability to implement data governance tools (Purview preferred), data security (One Lake Security preferred), and compliance requirements such as data retention schedules and audit.
• Specific…