Data Quality Engineer

JOB SUMMARY

At Vantage Bank, we are committed to supporting our customers, valuing our employees, embracing diversity, fostering meaningful connections, and providing outstanding service every step of the way. The Data Quality Engineer is a member of the Shared Services’ Analytics and Insights team responsible for implementing and maintaining data quality controls across Vantage Bank's lakehouse platform on Azure Databricks. The role designs rule‑based checks, monitors pipelines for anomalies and SLA breaches, and works directly with data stewards and business owners to translate data requirements into governed, auditable DQ rules stored as versioned artifacts.

The position requires technical depth in SQL, Python, and the Databricks platform alongside the ability to communicate findings and rule decisions to business stakeholders in plain language. The Data Quality Engineer applies AI tools to routine tasks such as test scaffolding, incident summarization, and remediation drafting, and is directly responsible for reviewing and validating all AI assisted outputs before use.

The role carries ongoing accountability for lineage documentation, rule catalog maintenance, and producing reproducible evidence in support of audit and regulatory requests.

• Build and maintain DQ monitoring dashboards and alerting pipelines that track data freshness, completeness, accuracy, conformity, and uniqueness across the lakehouse.
• Investigate data quality incidents, perform root cause analysis, coordinate remediation with upstream teams, and document incident summaries.
• Partner with data stewards and business owners to capture business rules, thresholds, and tolerances and implement them as versioned, testable DQ rule artifacts.
• Implement DQ checks in Databricks pipelines using SQL and Python, Lakeflow Spark Declarative Pipeline expectations, and the bank's internal data quality rules engine.
• Develop automated validation checks integrated into deployment workflows, including pipeline‑level tests at ingestion and transformation stages.
• Apply AI tooling to generate DQ test scaffolds, summarize incidents, propose remediation steps, and draft stakeholder communications; review and validate all outputs before use.
• Maintain data lineage, rule catalogs, and governance documentation in alignment with bank data governance policies.
• Support risk, audit, and regulatory requests by producing traceable rule histories and reproducible validation artifacts.
• Communicate data quality findings, SLA impacts, and rule change decisions to business stakeholders in business terms.
• Manage the review, rollout, and retirement of DQ rules across affected pipelines and coordinate rule change decisions with impacted teams.
• Analyze incident trends, identify systemic data issues, and propose structural improvements such as schema contracts and upstream data agreements.
• Stay current with Databricks platform capabilities relevant to data quality, including Delta Lake expectations, pipeline validation features, and observability tooling.

These specifications are general guidelines based on the minimum experience normally considered essential to the satisfactory performance of this position.

• Bachelor’s degree in computer science, Information Systems, Mathematics, or a related field; or 3+ years of directly relevant data quality or data engineering experience in lieu of a degree.
• 2 to 4 years of experience in data quality, data engineering, or a closely related data discipline.
• Proficiency in SQL and Python for writing validation logic, DQ checks, and automation scripts.
• Hands‑on experience with Azure Databricks including Lakeflow Jobs, Delta Lake, and pipeline development.
• Experience building or maintaining data quality rules within ELT or ETL pipelines.
• Working knowledge of core data quality dimensions: completeness, accuracy, conformity, freshness, uniqueness, and referential integrity.
• Familiarity with Git and deployment workflow basics for versioning and promoting rule artifacts across environments.
• Ability to communicate data findings and technical decisions clearly to both engineering peers and business stakeholders.
• Demonstrated ability to adopt AI tooling for automating DQ tasks, drafting rule code and test scaffolds, and summarizing incidents, with consistent human review of all outputs.
• Preferred: experience with data quality frameworks or observability platforms, such as Great Expectations, Monte Carlo, or comparable tools.
• Preferred: familiarity with Unity Catalog lineage, data catalog features, or similar governance tooling.
• Preferred: experience with schema contract standards or data contract patterns.
• Preferred: experience in a regulated financial services or banking environment.
• Preferred: familiarity with cloud data platforms, preferably Azure.
• Preferred: exposure to data governance frameworks and auditable metadata management practices.