Data Quality Engineer

We are seeking an experienced Data Quality Engineer to design, implement, and operationalize enterprise-scale data quality frameworks within a modern Databricks Lakehouse architecture. The ideal candidate will possess strong expertise in Databricks, PySpark, Delta Lake, SQL, and data governance, with a proven track record of embedding data quality controls throughout the data lifecycle. This role will be responsible for ensuring data integrity, reliability, compliance, and observability across large-scale cloud-based data platforms while driving quality-by-design principles across the organization.

• Design and implement enterprise-wide data quality frameworks aligned with Lakehouse architecture, including Bronze, Silver, and Gold data layers.
• Define, implement, and enforce data quality rules covering:
  • - Accuracy
  • - Consistency
  • - Timeliness
  • - Validity
• Develop reusable validation, reconciliation, profiling, and monitoring frameworks within Databricks environments.
• Establish automated data quality checks integrated into ELT and ETL pipelines.
• Embed quality controls directly into Databricks workflows, Spark processing pipelines, and Delta Lake architectures.
• Develop scalable validation processes supporting both batch and real-time data ingestion pipelines.
• Partner with Data Engineers to ensure quality gates are enforced across ingestion, transformation, and consumption layers.
• Optimize data quality processes for performance, scalability, and reliability across large distributed datasets.
• Implement and maintain data observability solutions, including dashboards, alerts, monitoring metrics, and reporting frameworks.
• Monitor data pipelines and proactively identify anomalies, failures, data drift, and data quality degradation.
• Lead root cause analysis (RCA) activities and drive resolution of data quality issues.
• Develop and maintain enterprise data quality scorecards and performance reporting.
• Ensure adherence to enterprise data governance standards, including metadata management, data lineage, traceability, and auditability.
• Collaborate with Data Governance teams to align data definitions, ownership models, and control frameworks.
• Support regulatory compliance requirements through auditable and repeatable data quality processes.
• Define and enforce data quality SLAs, standards, and data contracts across business domains.
• Implement CI/CD practices for data quality rules, monitoring processes, and validation frameworks.
• Automate testing and validation of data transformations, integrations, and pipelines.
• Develop reusable enterprise libraries and frameworks for scalable data quality enforcement.
• Partner with Data Architects, BI teams, Data Engineers, and business stakeholders to drive data quality initiatives.
• Provide technical leadership, mentorship, and best-practice guidance across teams.
• Serve as the subject matter expert (SME) for enterprise data quality strategies and standards.
• Drive continuous improvement and innovation in data quality methodologies, tools, and practices.

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, Data Science, or a related field.
• Proven experience in Data Engineering, Data Quality Engineering, or related data management roles.
• Strong hands-on experience with:
  • Databricks
  • Py Spark
• Extensive experience implementing enterprise data quality frameworks and controls within modern cloud data platforms.
• Advanced SQL development, data validation, and data profiling expertise.
• Experience integrating data quality processes into ELT/ETL pipelines and orchestration frameworks.
• Strong knowledge of data lifecycle management principles and best practices.
• Experience working with large-scale datasets in AWS or Azure cloud environments.
• Strong understanding of:
  • Data Governance
  • Metadata Management
  • Data Quality Controls
  • Auditability Requirements
• Experience implementing automated validation, reconciliation, and monitoring processes.
• Strong analytical, troubleshooting, and problem-solving skills.
• Ability to translate business requirements into scalable technical solutions.
• Excellent communication and stakeholder management skills.
• Experience working within Agile and Dev Ops delivery environments.

• Experience with in Financial Services, Banking, Insurance, or other regulated industries.
• Familiarity with Collibra or other Enterprise Data Governance platforms.
• Experience implementing data observability and monitoring solutions, including:
  • Deequ
• Experience with real-time or…