fix: make intensity levels genuinely differ via mode-gated rule blocks - #668
Open
SomSamantray wants to merge 7 commits into
Open
fix: make intensity levels genuinely differ via mode-gated rule blocks#668SomSamantray wants to merge 7 commits into
SomSamantray wants to merge 7 commits into
Conversation
Issue DietrichGebert#664: lite/full/ultra rendered ~96%-identical injected documents. Restructure the canonical SKILL.md so each level owns a gated rule block (advisory / enforced / deletion-first) around a shared ungated core, and graduate the shared filter from stateless line-drop to block selection. The block markers are render-invisible HTML comments (`<!-- mode: X -->`), stripped by the filter, and deliberately avoid the pinned `ponytail:` comment convention. The existing table-row/quoted-example line-drop is preserved as a fallback for non-block content, and the fallback path now carries a per-level enforcement line so a SKILL.md read failure no longer reproduces the near-identical levels. Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
…ty gate The Hermes plugin re-implements the skill filter in Python and had already diverged from the JS filter (its worked-example matcher lacked the quote requirement). Mirror the new block-selection state machine, add the quote requirement, and add a per-level stance line to the Python fallback so the failure path also diverges per level. A cross-language parity test now runs the JS and Python filters over the canonical SKILL.md for all three levels and asserts identical output, so the two implementations cannot silently fork again (KTD2, R7). Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
A `/ponytail lite|full|ultra` switch previously emitted only a one-line confirmation, so the running session kept enforcing the previous level's rules until the next session start. Now the switch re-injects the new level's full ruleset (prefixed by the MODE CHANGED header) on the hosts whose hook output carries context — Claude Code raw stdout and Codex hookSpecificOutput. Copilot drops all non-SessionStart output by design and applies the switch at the next session start, which stays documented. Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
Extend the rule-copy drift guard so the 9 safety invariants must be present in the ungated region of SKILL.md (not merely somewhere in the raw file, which would let a mode's filtered output silently lose a carve-out), and assert that no gated mode block leaks into AGENTS.md or its 7 compact copies (instruction-tier hosts are mode-less). Add a leak test asserting the raw SKILL.md and OpenClaw copy carry balanced, well-formed mode markers. Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
…full The three levels are now real behavior dials (lite advisory, full enforced, ultra deletion-first); update the help card, README, command files, and the agent-portability doc (instruction-tier hosts stay mode-less full). The benchmark arms and robustness audit now load the mode-filtered full ruleset (the production injection path) instead of the raw SKILL.md, which since DietrichGebert#664 carries the union of all three levels plus gating markers. Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
The block state machine only needs the `skip` flag; the block-mode label is assigned and immediately consumed. Simplify both the JS and Python mirrors (behavior-identical, pinned by the cross-language parity test), align the JS fallback's drop-only branches with the Python single-exit shape, and make the OpenClaw leak test resolve paths relative to the test file rather than cwd. Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
- lite: add an explicit override naming the suspended core rules so the advisory level no longer contradicts the enforced ungated core (P1) - Copilot: honor the persisted session flag at SessionStart so a mid-session switch really applies at the next session start, with a regression test (P2) - fallback: make the lite fallback advisory-only in both the JS and Python implementations so R9's per-level divergence is real, not cosmetic (P2) - drift guard: assert each mode block carries its distinctive phrase and not the others', so a swapped/rewritten block set fails the guard (P2) - leak test: make the marker validation order-sensitive (stack-based) so crossed or unclosed block sequences fail (P2) - parity test: add a blockless fixture exercising the retained stateless line-drop fallback across both languages (P2) - agentic benchmark: drop the unreachable _ponytail_prompt() fallback (P3) Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Switching ponytail between
lite,full, andultraused to change almost nothing: the three levels injected a near-identical ruleset (96 of 99 lines the same) that differed only in a stance sentence and one example. The levels are now real behavior dials.liteis advisory — build what's asked, name the lazier alternative, let the user pick, with an explicit override suspending the enforced core rules.fullstays the enforced default.ultrais deletion-first — challenge the requirement before adding.What changed
/ponytail lite|full|ultraswitch now re-injects the new level's ruleset on Claude Code and Codex instead of emitting only a one-line confirmation. Copilot (whose hook output is dropped by design) honors the persisted switch at the next session start.Fixes #664
Validation
npm test: 83/84 pass (the one failure is the pre-existing, unrelated CSV pandas check).node scripts/check-rule-copies.jspasses with the 9 invariants pinned to the ungated core.lite/full/ultradocuments confirms each level's exclusive rules are present and the other levels' are absent.New concepts
Mode-gated instruction blocks
A single canonical document carries per-mode content wrapped in render-invisible HTML-comment markers (
<!-- mode: lite -->...<!-- /mode: lite -->), and a small state-machine filter selects the active mode's block while stripping the markers.Why here: the alternative was three separate per-level documents, which would have tripled the surface every drift guard and copy must keep aligned. One document degrades gracefully for hosts that ingest it whole (the OpenClaw verbatim copy, skill pickers, benchmarks), and the markers are invisible when rendered.
The shipped filter walks the lines, skips the non-active blocks, and keeps the shared core; the leak test validates the marker sequence with the same stack logic.
When not to use it: when modes differ in more than rule text — if each mode needs different tooling or a different document shape, separate files are clearer.
Cross-language parity fixture
A test that runs two independent implementations of the same filter (the shared JS builder and the Hermes Python mirror) over the same fixture for every mode and asserts byte-identical output, so the two can never silently fork again.
Why here: the Hermes plugin re-implements the skill filter in Python, and it had already diverged from the JS version (its worked-example matcher lacked the quote requirement). A shared fixture is the only cheap guarantee that a future edit keeps both languages in lockstep.
When not to use it: when one implementation is generated from the other (a generator test is the right guard), or when the two implementations intentionally differ.