Lead Data Engineer

Description

Job Location:

The primary work location for this role is Raleigh, NC with a hybrid work model.

Envestnet is an adaptive Wealth Tech company that is redefining the future of wealth management by helping advisors meet the moment with its comprehensive technology, actionable insights, and industry leading support. Backed by over 25 years of experience and approximately $7.0 trillion in platform assets, Envestnet is trusted by over one third of financial advisors across leading banks, wealth managers, brokerages, and RIAs.

For a deeper look at how Envestnet is shaping the future of financial advice, visit

The Lead Data Engineer will play a critical implementation role on the Data Engineering and Data Services team and be responsible for data pipeline solutions design and development, troubleshooting, and optimization tuning on the next generation data and analytics platform being developed with leading edge big data technologies in a highly secure cloud infrastructure. The Data Engineer will serve as a liaison to platform user groups ensuring successful implementation of capabilities on the new platform.

The Lead Data Engineer will also take a lead role on functional teams or projects.

• Deliver end-to-end data and analytics capabilities, including data ingest, data transformation, data science, and data visualization in collaboration with Data and Analytics stakeholder groups.
• Design and deploy databases and data pipelines to support analytics projects.
• Develop scalable and fault-tolerant workflows.
• Clearly document issues, solutions, findings and recommendations to be shared internally & external.
• Learn and apply tools and technologies proficiently, including:
  • Languages:
    
    SQL (standard and DB-specific), Python, Scala, Bash
  • Frameworks:
    Hadoop, Spark, Kafka
  • Gen AI: RAG, Embedding, Agents, LLMs, Parameter Tuning
  • Cloud Computing: AWS
  • Tools/Products:
    Data Science Studio, Alteryx, Jupyter, Tableau, PowerBI
• Performance optimization for queries and dashboards.
• Develop and deliver clear, compelling briefings to internal and external stakeholders on findings, recommendations, and solutions.
• Analyze client data & systems to determine whether requirements can be met.
• Test and validate data pipelines, transformations, datasets, reports, and dashboards built by team.
• Develop and communicate solutions architectures and present solutions to both business and technical stakeholders.
• Provide end user support to other data engineers and analysts.
• Be a team leader and take lead role on functional teams or projects.
• Leads others to solve complex problems; uses sophisticated analytical thought to exercise judgment and identify innovative solutions.
• Interprets internal/external business challenges and recommends best practices to improve products, processes or services.
• Adherence to and application of Envestnet legal, compliance, risk, business continuity and administrative policy within the role and department(s) including the timely completion of training & awareness, affirmations and testing as requested.
• As part of the responsibilities for this role, you will understand and readily support Envestnet's established corporate business practices, policies, internal controls and procedures designed to create value or minimize risk.

• 8-12 years of relevant experience or equivalent combination of experience and education.
• Expert experience in the following:
  • SQL, Python, PySpark. Other programming languages (R, Scala, SAS, Java, etc.) are a plus.
• Data and analytics technologies including SQL/No
  
  SQL/Graph databases, ETL, and BI.
• Knowledge of CI/CD and related tools such as Gitlab, AWS Code Commit, etc.
• AWS services including EMR, Glue, Athena, Batch, Lambda Cloudwatch, DynamoDB, EC2, Cloud formation, IAM and EDS.
• Solid scripting skills (e.g., bash/shell scripts, Python).
• Proven work experience in the following:
  • Data streaming technologies
  • Using LLMs, creating RAG, choosing vector vs graph db for embedding, etc.
  • Big Data technologies including, Hadoop, Spark, Hive, Teradata, etc.
  • Linux command-line operations.
  • Networking knowledge (OSI network layers, TCP/IP,…