Senior Data Engineer - IoT

Schedule: Monday–Friday | Primarily On-Site with flexibility for 1–2 days remote

Compensation: $185,000 + bonus

Our client is seeking a Senior Data Engineer – IoT to take a leading role in the development and long-term direction of its IoT data platform. This is a newly created position for a highly hands‑on engineer who brings significant experience working with large‑scale IoT, machine, or telemetric data.

This individual will help drive the continued migration of IoT data into Databricks, while designing and building a scalable data platform that supports real‑time and near‑real‑time reporting, analytics, data science, AI/ML, customer‑facing applications, and sustainability initiatives.

This is an opportunity to have meaningful ownership over the organization's IoT data strategy—not simply implement an existing specification. The ideal candidate will be comfortable moving between hands‑on engineering, architecture, technical guidance, and business‑facing discussions.

• Design, build, and maintain end‑to‑end IoT data pipelines from edge devices into Databricks using Bronze/Silver/Gold medallion architecture.
• Lead the migration of IoT data from a third‑party application into a scalable Databricks environment.
• Develop and optimize streaming ingestion pipelines using technologies such as Apache Kafka, Azure Event Hubs, and Databricks Structured Streaming.
• Establish data engineering best practices, including pipeline design patterns, data quality frameworks, schema governance, and scalable development standards.
• Architect and optimize Delta Lake tables, including partitioning, Z‑ordering, and data lifecycle management for performance and cost efficiency.
• Collaborate with infrastructure and cloud teams on Databricks clusters, Unity Catalog, Delta Live Tables, and related cloud technologies.
• Develop solutions capable of supporting multi‑terabyte, high‑volume telemetry and time‑series data.
• Partner with data scientists, analysts, and business stakeholders to translate requirements into scalable data solutions.
• Ensure IoT data is structured and optimized for downstream analytics, AI/ML, reporting, and customer‑facing use cases.
• Mentor junior and mid‑level data engineers through code reviews, pair programming, technical guidance, and architectural direction.
• Troubleshoot production pipeline issues, perform root‑cause analysis, and implement long‑term solutions.
• Evaluate emerging technologies and recommend improvements to the organization's IoT and data engineering platform.
• Represent the data engineering team in technical and business discussions and help shape the organization's long‑term IoT data strategy.

• 5+ years of hands‑on experience working with IoT, machine, telemetric, or similarly large‑scale device data.
• Strong senior‑level data engineering experience, preferably in environments involving high‑volume or real‑time data.
• Strong proficiency in Python and SQL, with significant experience developing production‑grade Spark/PySpark solutions.
• Experience designing and implementing data pipelines for high‑frequency telemetry, time‑series data, or device‑generated data.
• Experience with Databricks and/or other large‑scale cloud data platforms.
• Strong understanding of data architecture, pipeline design, data quality, and schema governance.
• Experience working with streaming or near‑real‑time data architectures.
• Ability to lead technical discussions and translate business requirements into scalable technical solutions.
• Strong communication and interpersonal skills, with the ability to work effectively with engineers, data scientists, analysts, and business stakeholders.
• Ability to provide technical guidance and mentorship to other engineers.