QA Cyber Analyst

The QA Cyber Analyst evaluates the quality, accuracy, consistency, and defensibility of app review decisions. While App Review reviewers assess submitted applications for functionality, policy compliance, privacy, safety, and platform requirements, the QA Analyst evaluates the quality of those reviews — ensuring reviewers apply the correct policies and review criteria, perform sufficient testing or evidence review, document their findings clearly, and reach decisions supported by the facts of the case.

The QA Analyst maintains a high and consistent review standard by identifying reviewer errors, missed risks, unsupported decisions, documentation gaps, policy interpretation issues, and recurring quality trends, and requires strong analytical judgment, policy fluency, attention to detail, technical curiosity, and the ability to distinguish reviewer error from ambiguity in policy, process, tooling, or available evidence.

• Audit completed app review cases against established QA standards and scoring rubrics.
• Assess whether the reviewer selected and applied the correct review checks, policies, and escalation paths.
• Determine whether the reviewer's final decision was accurate, complete, and supported by available evidence.
• Classify quality issues by severity, including decision-impacting errors, process errors, documentation gaps, and coaching opportunities.
• Identify both under-enforcement (missed policy or safety issues) and over-enforcement (unsupported rejections or unnecessary escalations).

• Review submitted app metadata, developer-provided materials, reviewer notes, screenshots, testing evidence, tool information, policies, privacy disclosures, and relevant case context.
• Determine whether the reviewer gathered and documented enough evidence to support their conclusions.
• Identify missing, contradictory, irrelevant, or insufficient evidence.
• Evaluate whether another reviewer could reasonably understand and reproduce the original decision based on the documented record.
• Perform targeted validation where required by the QA workflow, without assuming every case requires a full second review.

• Validate whether reviewers correctly interpreted and applied app review policies and platform requirements.
• Assess reviewer handling of privacy, data access, user safety, regulated or high-impact domains, impersonation, misrepresentation, third‑party access, and other trust‑related risks.
• Ensure reviewers evaluate the app's actual functionality and submitted evidence rather than relying on unsupported assumptions or hypothetical misuse.
• Identify cases where ambiguity should have led to escalation rather than approval or rejection.
• Surface unclear policy guidance or recurring interpretation gaps.

• Evaluate whether reviewers sufficiently assessed the app's core workflows, tool behavior, authentication requirements, MCP server connectivity, OAuth flows, test account access, and developer‑provided testing instructions.
• Determine whether technical blockers were reasonably investigated before being treated as review‑limiting.
• Identify cases where an app was approved despite meaningful technical failures or incomplete testing.
• Distinguish between true app issues, reviewer execution errors, incomplete submission materials, and tooling limitations.

• Support internal and client‑facing calibration sessions to align on review standards, QA scoring, defect severity, and policy interpretation.
• Present QA findings, trends, and examples clearly to client stakeholders during calibration discussions.
• Explain the rationale behind QA decisions in a balanced, evidence‑based way,…