Skip to content

Implement evidence-packet and sufficiency-rubric tooling from interface program #18

Description

@mdheller

Purpose

Turn the Einstein–Heller typed-interface benchmark into executable governance tooling.

Context

The interface program distinguishes latent state, trace maps, semantic lifts, reconstruction, and sufficiency. Model cards, system cards, provenance bundles, and adjudication worksheets are typed governance interfaces: they are not full microstate dumps, but can be semantically or governance sufficient for specific claims.

Deliverables

  • docs/interface-sufficiency-rubric.md
  • docs/evidence-packet-method.md
  • schemas/evidence-packet.schema.json
  • schemas/adjudication.schema.json
  • examples/einstein_heller_frozen_evidence_packet_v2_1.csv
  • examples/einstein_heller_frozen_evidence_packet_v2_1.json
  • examples/einstein_heller_adjudication_template_v2_1.csv
  • src/ or tools/ implementation for scoring vectors and conservative collapse
  • tests for documentation vectors and service-surface vectors

Minimum semantics

  • Raw field code is tri-state: 0, U, 1.
  • Conservative binary collapse maps 1 -> 1 and {0,U} -> 0.
  • Positive governance-ready calls require all required governance fields to be explicit 1, not inferred through ambiguity.
  • Output must distinguish semantic sufficiency from microstate sufficiency.

Acceptance criteria

  • CLI or script can score a frozen evidence packet deterministically.
  • Tests cover OLMo-header-style task sufficiency, full-card governance sufficiency, intermediate documentation-partial cases, and service-card vector scoring.
  • Output includes class, missing fields, Hamming margin, and sufficiency label.
  • No benchmark claim is promoted beyond single-team frozen packet unless independent rater data exists.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions