Skip to content

Add review severity levels and explicit override records #11

Description

@tomvaillant

Problem

Render-review concerns are currently advisory strings. Source, accessibility, title-fidelity, or data-fidelity failures can be recorded and then shipped with the same “ship it” action as a minor stylistic observation.

This collapses materially different risks into one bucket and provides no durable evidence that a journalist knowingly overrode a serious finding.

@rmdms

Suggested solution

Replace string-only concerns with structured findings:

{
  "id": "source-traceability",
  "criterion": "source",
  "severity": "blocking",
  "status": "open",
  "summary": "",
  "evidence": [""]
}

Use three levels:

  • Blocking: claim/data mismatch, fabricated or invalid attribution, serious a11y failure, broken interaction required by the format, stale/unproven artifact, or another defect that makes the visual unsafe to ship.
  • Warning: a real editorial limitation that may be acceptable with informed judgment.
  • Informational: non-blocking improvement or context.

gate-render should refuse while blocking findings remain open. If policy permits an override, require an explicit journalist action that records finding ID, reason, timestamp, actor label, and the exact artifact hash being overridden. Re-production invalidates overrides tied to the prior artifact.

Warnings should require visible acknowledgement but not the same override ceremony. Informational findings should remain in the review record without interrupting approval.

Define the severity mapping centrally so the same defect cannot be blocking in one producer and advisory in another.

Acceptance criteria

  • Review findings use a validated structured schema with stable IDs.
  • Blocking findings prevent approval until resolved or explicitly overridden under policy.
  • Every override is attached to the finding and current artifact hash with a recorded reason.
  • Re-production resets findings/overrides for the new artifact.
  • Journalist-facing preview clearly separates blockers, warnings, and information.
  • Tests cover severity mapping, acknowledgement, allowed/disallowed overrides, artifact invalidation, and export refusal.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions