Senior Data Engineer

Employment Type: Fulltime Direct Hire no C2C or sponsorship available

Talent Groups is seeking a Senior Data Engineer to join a growing Enterprise Data team and play a key role in building and scaling modern data platforms. This role focuses on designing and optimizing data pipelines that support analytics reporting and machine learning initiatives across the organization. You will work closely with technical teams and executive stakeholders while serving as an onsite technical leader.

This is a hands on senior level role with a strong emphasis on Python, PySpark, Databricks, and Azure. The team is currently in the midst of an Azure migration creating an opportunity to make a meaningful impact and help shape future state architecture and best practices.

• Design develop and optimize scalable data pipelines using Azure Databricks
• Build and manage data ingestion transformation and storage solutions using Azure Data Factory, Azure Data Lake, and Delta Lake
• Develop data solutions that support analytics reporting and machine learning use cases
• Implement CI CD pipelines using tools such as Azure Dev Ops, Git, and Terraform
• Optimize Spark performance including partitioning orchestration and cost efficiency
• Collaborate with analysts data scientists and business stakeholders to deliver trusted reusable datasets
• Serve as the onsite face of the data engineering team and partner with leadership
• Mentor junior and associate data engineers
• Monitor troubleshoot and improve the reliability and scalability of existing pipelines

• Five plus years of experience in data engineering with a strong focus on Azure
• Hands on experience with Azure Databricks, Azure Data Lake, and Data Factory
• Strong proficiency in PySpark, SQL, Python, and Apache Spark
• Solid understanding of data warehousing, ETL, ELT, and data modeling best practices
• Experience with version control CI/CD pipelines and infrastructure as code
• Knowledge of Spark performance tuning and job orchestration
• Strong problem solving skills and attention to detail
• Excellent communication skills with both technical and non technical stakeholders
• Ability to work independently and lead in a fast paced Agile environment
• Passion for delivering clean maintainable high quality code

• Experience with Unity Catalog, Databricks Workflows, and Delta Live Tables
• Familiarity with Dev Ops practices or Terraform for Azure provisioning
• Understanding of data security, RBAC, and cloud compliance
• Experience integrating Databricks with Power BI or similar analytics tools
• Exposure to real time data processing using Kafka Event Hubs or Structured Streaming

Talent Groups would like to thank all applicants. Only those who qualify for an interview will be contacted.

Talent Groups is an equal opportunity employer.