Skip to content

memory:promote — production-vs-fixture-precision-fix - #772

Open
github-actions[bot] wants to merge 2 commits into
mainfrom
memory/production-vs-fixture-precision-fix
Open

memory:promote — production-vs-fixture-precision-fix#772
github-actions[bot] wants to merge 2 commits into
mainfrom
memory/production-vs-fixture-precision-fix

Conversation

@github-actions

@github-actions github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Summary

What this lesson captures

Issue #742 (3x-disjoint cell) and Issue #744 (6x-confusable cell, see
[[no-hedge-single-peer-directive]]) are sibling -13.3% precision
regressions, not duplicates — and they were fixed at two genuinely
different loci:

The lesson generalizes: trace a routing regression to its cause before
picking a fix locus — production tool descriptions when the bias is generic
(any worker can hedge/under-delegate), the eval fixture prompt when it's
specific to that eval's synthetic mesh.

It also flags two things PR #748 shipped with, per its own reviewer:

  1. Opposing-instruction tension — the new list_peers text ("don't
    guess in favor of a loosely-matching peer") and the new
    delegate_to_peer text ("delegate even if you think you know the
    answer") pull in opposite directions for a loosely-matching task, risking
    swapping a precision fix for a recall regression (or vice versa) on a
    different eval cell.
  2. No behavioral verification — the only new test asserts the
    description strings landed (assert.match), not that routing behavior
    actually changed. Confirming the regression is really fixed needs a real
    scripts/eval-perf.mjs re-run on the 3x-disjoint cell, checking precision
    and recall together.

Cross-linked [[no-hedge-single-peer-directive]]
[[production-vs-fixture-precision-fix]] so either doc surfaces the other.

Validation

  • node scripts/validate-quick-memory.mjs dev-mesh/curator/memory/quick.jsonvalid (no quick.json changes; workflow files only)
  • quick.json entry count: 200/200 (unchanged — already at the hard cap; this lesson was routed to workflows/ instead for exactly that reason)

Review checklist

  • Lesson is durable and general enough for future recall
  • No code changes (memory root only)
  • quick.json caps respected (200/200 entries, unchanged)
  • Validator passed

…fix)

Precision regressions in mesh-scan routing evals can be fixed at two
distinct loci: the real production tool descriptions in
src/a2a/peer-bridge.js (affects every deployed agent) or the eval
harness's synthetic caller prompt (affects only that fixture). PR #748
fixed the former for Issue #742 (3x-disjoint cell); the sibling
no-hedge-single-peer-directive lesson fixed the latter for Issue #744
(6x-confusable cell). Cross-linked both docs, and flagged PR #748's
reviewer-noted risk: its two added tool-description directives pull in
opposite directions (don't-guess vs. must-delegate) and were verified
only by a substring-match unit test, not a real eval-matrix re-run.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

Triager classificationsrc/dev-mesh/classify.js precedence (infra > out-of-scope > flake > real_bug)

  • label: infra_auth
  • reason: no test/job ever executed. Both CI (run 28593349765) and dev-mesh-review completed with action_required and zero jobs — this is a GitHub Actions approval gate (triggering actor github-actions[bot], pull_request event withheld pending manual approval), not a code failure. There is no job log to read because nothing ran (ranAnyTest: falseinfra_auth per the classifier, same branch as an auth/secret/network signature).
  • action: escalate to a human — do not fix or re-kick. A re-run without approval will land back in action_required; someone with write access needs to approve the run from the Actions tab (or branch protection needs to stop requiring these checks for bot-authored PRs). This is already tracked as needs-human in needs-human: PR#772 stuck (not-clean:UNSTABLE) — non-required check unmet #778, so no new issue filed — flagging here as the classified checkpoint per the triage sweep.

No code change proposed; I don't edit code from this role.

PR #772 stuck in mergeStateStatus=UNKNOWN for 3752min.
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

🤖 dev-mesh janitor: pushed an empty commit to nudge GitHub's mergeable recalc. If the state stays UNKNOWN after a few minutes, this PR likely needs a human.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants