Senior Data Engineer on-site

Senior Data Engineer (on-site) – Plano, TX

We are looking for a highly skilled and experienced Senior Data Engineer with deep expertise in Azure Databricks to contribute to the design, development, and optimization of our modern data platform. In this role, you will be responsible for building scalable ETL pipelines, implementing data lakehouse architectures, and supporting advanced analytics use cases in a high-volume data environment. You will work closely with data architects, data scientists, business stakeholders, and cloud infrastructure teams to deliver reliable, secure, and high-performance data solutions.

• Design and implement scalable ETL/ELT pipelines using Azure Databricks (PySpark, Spark SQL).
• Build robust data ingestion workflows from diverse sources (Event Hub, APIs, Azure SQL, Blob Storage, etc.) into the Bronze/Silver/Gold layers of the medallion architecture.
• Define and implement best practices for Delta Lake and Unity Catalog for schema evolution, versioning, and governance.
• Optimize performance and cost efficiency of large-scale data pipelines through intelligent partitioning, caching, clustering, and tuning Spark configurations.
• Integrate Databricks with Azure Data Lake Storage (ADLS Gen2), Azure Data Factory (ADF), Azure Synapse, Event Hub, Key Vault, and Azure Dev Ops.
• Implement secure and governed access to data using managed identities, RBAC, and data masking techniques.
• Partner with data analysts, data scientists, and business users to understand data needs and deliver trusted datasets and KPIs.
• Provide technical leadership and mentorship to junior engineers, conducting code reviews and guiding architectural decisions.
• Collaborate with enterprise architects and Info Sec to ensure compliance with data security and privacy policies.
• Implement data quality checks, alerts, and lineage tracking using tools such as Great Expectations, Unity Catalog, and Data Observability platforms.
• Set up monitoring dashboards, alerts, and job recovery mechanisms for mission-critical pipelines.

• Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering, with 3+ years focused on Azure Databricks and Spark (PySpark).
• Expertise in Delta Lake, Unity Catalog, Databricks Notebooks, and SQL Warehousing.
• Hands‑on experience with Azure cloud services: ADLS Gen2, Event Hub, ADF, Key Vault, Synapse.
• Strong programming skills in Python, SQL, and familiarity with Scala.
• Solid understanding of data modeling (OLTP/OLAP), star/snowflake schemas, and performance tuning.
• Experience with CI/CD pipelines in Databricks using Databricks CLI, Terraform, or Azure Dev Ops.

• Databricks certification (e.g., Databricks Certified Data Engineer Professional).
• Experience with real‑time streaming (e.g., Structured Streaming, Event Hub, Kafka).
• Familiarity with MLflow or machine learning pipeline support in Databricks.
• Experience with Power BI, Azure Purview, or Microsoft Fabric for downstream reporting.
• Knowledge of data governance, lineage, and compliance standards (GDPR, HIPAA, SOC2).

Ziosk is an Equal Opportunity employer offering competitive benefits and compensation. Candidates must be eligible to work in the U.S. and be able to commute to Plano, TX daily. Applicants must be authorized to work for any employer in the U.S. No agencies or third-party recruiters, please.