Data Assurance Lead

Overview

Deep Light AI is a specialist AI and data consultancy with extensive experience implementing intelligent enterprise systems across multiple industries, with particular depth in financial services and banking. Our team combines deep expertise in data science, statistical modeling, AI/ML technologies, workflow automation, and systems integration with a practical understanding of complex business operations.

We are looking for a Data Assurance Lead to serve as the ultimate guardian of data trust across our clients' complex Lakehouse environments. In this high-impact consultancy role, you won't just monitor data; you will design, implement, and operationalize a robust Data Assurance Framework from the ground up. Utilizing Soda Core across the Medallion architecture (Bronze, Silver, Gold), you will define rigorous quality rules, automate data contracts, and integrate scorecards into Open Metadata.

As a consultant, you are much more than a technical expert; you are a strategic advocate for data reliability. You must possess the communication mastery to "sell" the value of data assurance to stakeholders, lead cross-functional squads through aggressive migration targets, and embed a culture of quality within AWS Glue and Data Factory workflows. If you have 8+ years of experience—ideally within the Financial Services or Banking sectors—and thrive in fast-paced, Agile environments where you can automate the complex and influence the senior-most stakeholders, this is your next challenge.

• Data Quality Framework Implementation
  • Implement Soda Core data quality checks across all Medallion layers (Bronze, Silver, Gold).
  • Define and maintain a data quality rules library, covering:
    • Schema validation (completeness, uniqueness, data types)
    • Business rule validation (referential integrity, value ranges)
    • Reconciliation checks (source-to-target, row counts, checksums)
    • Freshness and timeliness SLAs as per data contracts
• Automation & Governance
  • Implement automated data contracts using Soda and secure sign-off with producers and consumers.
  • Create automated validation and reconciliation processes.
  • Establish data quality scorecards in Open Metadata, integrated with Soda.
  • Retrofit assurance checks across all live data flows.
  • Develop automated regression test suites for critical flows.
• Monitoring & Incident Management
  • Configure data quality alerts and thresholds.
  • Define escalation procedures for data incidents.
  • Collaborate with operational teams for alerting and monitoring.
  • Analyze data issues, implement countermeasures, and optimize check performance.
• Collaboration & Training
  • Work closely with Data Factory squads to embed data quality best practices.
  • Provide QA on Glue jobs to ensure compliance with the assurance framework.
  • Maintain comprehensive documentation and runbooks.
  • Train teams on data quality standards and tools.

As an AI consultancy, our greatest asset is the expertise of our people.

While technical mastery is the foundation of what we do, the ability to bridge the gap between complex data science and actionable business value is what defines your success with Deeplight.

We're looking for individuals who are not only world-class in their fields of specialism, but also compelling communicators and persuasive advocates for their own skills.

You will be the face of our firm, tasked with building trust, articulating the "why" behind your technical decisions, and effectively "selling" your vision to high-level stakeholders.

If you thrive on the challenge of presenting cutting-edge solutions as much as you do on building them, you will fit right in.

To be successful in this role, you should have:

• strong experience with data quality frameworks and tools (Soda Core or similar).
• hands-on experience with Lakehouse architectures and Medallion design.
• proficiency in AWS services (S3, Glue, Athena) and data validation techniques.
• Strong analytical and problem-solving skills.
• 8+ years experience in data engineering or data quality roles, with at least 3 years in data assurance leadership.

• identifying ways to automate their work / repetitive tasks.
• dealing with data contracts, metadata management, and…