Data Engineer

THE POSITION

You’ll be a key member of Vax Care’s Product Group, joining our Data Engineering team and reporting to our Data Engineering Lead. We are seeking a motivated and capable Data Engineer to join our team. As a Data Engineer, you will contribute to the design, development, and management of our data processing and analytics infrastructure. The ideal candidate will have hands‑on experience working with Spark and Databricks, a solid foundation in data engineering principles, and a desire to grow into a senior technical contributor.

• Develop and maintain Delta Lake‑based data pipelines using Databricks Workflows, Delta Live Tables (DLT), and Unity Catalog for enterprise data governance
• Build ELT/ETL pipelines using medallion architecture (bronze/silver/gold layers) supporting both batch and streaming workloads with Auto Loader and Structured Streaming
• Implement lakehouse solutions leveraging Delta Lake ACID transactions, Z‑ordering, liquid clustering, and partitioning strategies
• Support CI/CD pipelines for data workflows using Git integration and Databricks Asset Bundles
• Contribute to data quality frameworks using Delta Live Tables expectations and custom PySpark validation with automated alerting and SLA monitoring
• Create materialized views and incremental refresh strategies for optimized query performance
• Collaborate with data scientists, ML engineers and analysts to support feature engineering pipelines and MLOps workflows
• Participate in code reviews and contribute to technical design discussions
• Implement data observability, monitoring using Databricks SQL, Lakeview dashboards, and alerting frameworks
• Support cost optimization efforts leveraging Photon engine, serverless compute, and platform best practices
• Troubleshoot and resolve issues related to distributed computing, data skew, and performance bottlenecks
• Contribute to technical documentation including data contracts, runbooks, and data catalog metadata in Unity Catalog
• Follow Data Ops best practices including testing strategies, performance tuning, and data platform engineering principles
• Stay current with lakehouse architecture trends and emerging technologies to continuously improve our data infrastructure

• Bachelor’s degree in Computer Science, Data Engineering, Engineering, or related technical field OR equivalent practical experience
• Master’s degree or relevant industry certifications (Databricks Certified Data Engineer Associate, Azure Data certifications) are a plus

• Must be located in the Greater Orlando / Boston area.
• 3‑5 years of data engineering experience with 1+ years hands‑on production experience building data pipelines on Databricks and Apache Spark
• Experience contributing to lakehouse architecture implementations

Programming &

Languages:

• Strong proficiency in Python (PySpark, pandas) and SQL (complex queries, window functions, CTEs, query optimization)
• Experience with Spark SQL, Delta Lake SQL, and Databricks SQL

• Working knowledge of Apache Spark including:
• Performance fundamentals (partitioning, broadcast joins, data skew handling, caching strategies)
• Delta Lake features (ACID transactions, time travel, MERGE operations, CDC, liquid clustering)

• Hands‑on experience with Databricks including:
• Delta Live Tables (DLT) for declarative pipeline development
• Unity Catalog for data governance, access control, and lineage tracking
• Databricks Workflows and orchestration
• Basic understanding of cluster configuration and cost‑aware compute selection
• Databricks SQL and Lakeview dashboards

• Solid understanding of data modeling techniques:
• Dimensional modeling (star schema, fact/dimension tables)
• Medallion architecture (bronze/silver/gold layers)
• Slowly Changing Dimensions (SCD) implementations
• Strong SQL skills including query optimization and performance tuning
• Familiarity with modern lakehouse patterns and understanding of lakehouse vs. traditional data warehouse trade‑offs

• Familiarity with Dev Ops/Data Ops practices:
• Git workflows (branching strategies, pull requests, code reviews)
• CI/CD pipelines for data workflows (Git Hub Actions, Azure Dev Ops, Jenkins)
• Testing strategies (unit tests, integration tests, data quality tests)
• Basic monitoring and observability (logging, alerting)

• Works independently to deliver high‑quality, well‑tested solutions with meaningful impact on the team’s data infrastructure
• Takes ownership of assigned projects and drives them to completion with minimal oversight
• Strong communication and collaboration skills in cross‑functional team environments
• Proactive in identifying problems and proposing solutions, even outside immediate area of responsibility
• Demonstrates initiative in expanding technical depth and breadth, with a trajectory toward senior‑level engineering
• Open to feedback and committed to continuous improvement