Principal Software Engineer Data Platform Engineering
Listed on 2026-06-18
-
Software Development
Data Engineering
08/19/2025
Full Time
Active
Job Description:We are enabling the transformation of how energy and water are managed. As part of this mission, we are investing in cloud-native data platform technologies that power intelligent analytics at scale — particularly for large IoT data workloads. We are looking for a Principal Software Engineer - Data Platform Engineering with a strong background in data engineering to help design, build, and scale our next-generation data platform and services.
This role will focus on developing performant, scalable, and secure data analytics and data pipelines using Python, Apache Spark, Databricks, and Azure-native technologies. The ideal candidate is passionate about working with large-scale datasets, understands Spark query execution plans, and has experience deploying infrastructure using Terraform and container technologies like Docker.
Duties & Responsibilities:
- Design and implement scalable data pipelines for high-volume IoT telemetry data using PySpark, Spark SQL, and Databricks
- Analyze and optimize Spark jobs using execution plans, caching strategies, and memory tuning
- Develop Python-based micro-services and reusable libraries to support data processing workflows
- Work in a Dev Ops-driven Agile/Scrum environment to deliver high-quality code in fast iterations
- Collaborate with product managers, data scientists, and software engineers to define system requirements and data integration needs
- Build and manage infrastructure as code using Terraform to deploy workloads on Azure
- Create CI/CD pipelines for deploying and testing data workflows and services
- Package and deploy applications in Docker containers and manage runtime environments
- Apply strong testing practices, including unit tests, integration tests, and test automation
- Participate in architectural discussions and technical design reviews
- Ensure secure, reliable, and cost-effective operation of data workloads on cloud platforms
Required
Skills & Experience:
- Minimum of 10 years of software engineering experience, with at least 3 years focused on data platform or data engineering roles
- Proficient in Python, with strong knowledge of PySpark and Apache Spark
- Hands-on experience with Databricks and Spark SQL
- Strong understanding of Spark performance tuning and query execution plans
- Experience writing infrastructure-as-code using Terraform
- Working knowledge of Docker and containerized deployment
- Experience with CI/CD pipelines, version control (e.g., Git), and automated testing
- Experience with Agile/Scrum methodologies
- Bachelor’s degree in Computer Science, Engineering, or related field — or equivalent experience
Preferred
Skills & Experience:
- Experience with large-scale IoT or telemetry data processing
- Familiarity with Kafka, Event Hubs, or other streaming platforms
- Familiarity with cost optimization in cloud-based data processing
- Experience with monitoring and observability tools like Prometheus, Grafana, or Datadog
* This field is required Please enter valid email
Id.
Cell phone
* This field is required Please enter valid cell phone.
First Name
* This field is required Please enter valid first name.
Last Name
* This field is required Please enter valid last name.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).