Quality Analytics Lead
Listed on 2026-05-31
-
IT/Tech
Data Analyst, Data Scientist, Data Science Manager -
Quality Assurance - QA/QC
Data Analyst
Job Responsibilities
The Quality Analytics Lead is the dedicated technical resource bridging Welo Data’s Analytics and Quality organizations. Sitting within the Analytics team, this senior IC partners enterprise-wide with Quality Managers, Analysts, and leadership to design and maintain the data models, measurement frameworks, and analytical infrastructure that power evidence‑based quality decisions across programs and regions. At its core, this is an analytics engineering role.
The primary responsibility is building and owning the quality data layer — the dbt models, data marts, and Python‑driven modeling that transform raw operational data into a trusted, well‑documented foundation the Quality organization can rely on. Experimentation, stakeholder consulting, and BI delivery are all extensions of that foundation, not parallel tracks. The ideal candidate combines deep fluency in modern data modeling with a genuine understanding of quality operations, AI training data workflows, and experimental design.
They ensure the analytical systems they build directly improve how quality teams detect issues, validate improvements, and demonstrate impact to clients and leadership.
Build, maintain dbt models and data marts that serve the Quality organization’s enterprise reporting needs — covering throughput, accuracy, defect rates, CAPA effectiveness, annotator/rater performance, and program‑level quality health. Use Python for higher‑order data modeling tasks including cohort analysis, performance trend modeling, and custom aggregations beyond standard SQL/dbt scope. Partner with data engineers to define source data requirements, document data lineage, and ensure quality data is reliable, consistent, and analytics‑ready.
Own the quality analytics data layer end‑to‑end: from raw operational inputs to clean, tested, well‑documented marts consumed by dashboards, reports, and ad‑hoc analyses. Apply dbt testing, documentation, and best practices to build a trusted, maintainable codebase that scales as new programs and data sources are onboarded.
Collaborate with Quality Managers and Analysts to define, standardize, and operationalize quality metrics — including accuracy rates, defect categorization, sampling coverage, inter‑rater agreement, and CAPA closure effectiveness — consistently across all programs. Design measurement frameworks aligned to acceptance criteria and quality thresholds, ensuring metrics faithfully reflect program health and client commitments. Support rubric and guideline effectiveness measurement, helping quality teams understand whether their standards produce consistent, measurable outcomes across annotators and raters.
Champion data quality governance within the Quality org: own metric definitions, threshold documentation, and analytical methodology standards to reduce inconsistency and reporting variance. Define enterprise‑level quality dashboards in partnership with BI resources, translating mart output into clear, decision‑ready views for Quality Managers through to senior leadership. Analyze patterns in model evaluation outcomes, annotator disagreement, and guideline interpretation to surface systemic issues in AI training data and evaluation processes.
Design and execute A/B tests and controlled experiments to measure the impact of quality interventions, process changes, and annotator training programs — applying proper power analysis, significance testing, and results interpretation. Build success validation frameworks to confirm that CAPA actions and process improvements produce measurable, sustained outcomes — not just short‑term fluctuations. Develop performance attribution models that quantify the contribution of specific quality initiatives to outcome improvements, separating causal signal from noise in program performance trends.
Apply statistical methods to sampling design, audit analysis, and error pattern detection, surfacing systemic quality issues and their root causes with data‑backed evidence. Conduct pre/post analyses for…
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).