Skip to content

fix(review): say where GitHub shows a review's verdict, so the body opens with a finding - #1362

Merged
max-sixty merged 3 commits into
mainfrom
comment-verdct
Sep 21, 2026
Merged

max-sixty merged 3 commits into
mainfrom
comment-verdct

Conversation

@max-sixty

Copy link
Copy Markdown
Owner

Self-review COMMENTs open with "Reviewing my own PR, so this is a COMMENT rather than a verdict." (PRQL/prql#6352 has it in three rounds). No instruction contains or asks for the line. The first round produced it by chance; in 12 replays of that session it came up in none. Later rounds copy it, because each reads the earlier bot reviews in full for dedup and follows their form.

The Submit step now says GitHub labels the review approved or commented above its body, so the body opens with the first finding. It applies to every review: naming the self-authored case in the prompt raised the rate in a synthetic A/B. Replaying that PR's three review sessions, cut just before each wrote its body:

Skill text Round 2 (one earlier copy) Round 3 (two earlier copies)
Before 4/4 4/4
This PR 1/8, moved down from the opening 7/7

Nothing tried stops round 3, including an explicit prohibition: once two posted reviews carry a line, the session copies it.

The second commit backticks the event names (COMMENT, APPROVE) in skill prose, to match the review states (APPROVED, CHANGES_REQUESTED) already backticked beside them. On its own it doesn't change the opener (round 2 stayed at 4/4).

This was written by Claude Code on behalf of max-sixty

🤖 Generated with Claude Code

max-sixty and others added 3 commits September 21, 2026 11:30
…th a finding

Self-review COMMENTs across consumers open with "Reviewing my own PR, so
this is a COMMENT rather than a verdict" (earlier, "...an approval"). No
instruction contains the phrase. The review skill frames every review as
a verdict, and on a self-authored PR, where APPROVE is off the table, a
session sometimes spends the opening line explaining the missing one.
Once posted, the line spreads: each later round reads the earlier bot
reviews in full for dedup and mirrors their form.

Replaying the three PRQL/prql#6352 review sessions, cut just before each
wrote its body (4 runs per arm), the opener appeared 0/4, 4/4 and 4/4 in
rounds 1-3. With a sentence noting that GitHub labels the review
approved or commented above the body, so the body opens with the first
finding, round 2 kept the line in 1 of 8 runs, moved down from the
opening; round 3, which had two earlier reviews to copy, kept it in 7/7.
Naming the self-authored situation in the prompt raised the rate in a
synthetic A/B, so the sentence sits in Submit and applies to every
review.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e them

The review states (`APPROVED`, `CHANGES_REQUESTED`, `DISMISSED`) were
already code-formatted in skill prose; the event names a session submits
(`COMMENT`, `APPROVE`) were bare caps in 19 places. Both are API values,
so they now read the same.

This doesn't change the self-review opener: replaying the PRQL/prql#6352
review sessions, round 2 kept "so this is a COMMENT rather than a
verdict" in 4/4 runs with backticks alone, and backticks on top of the
Submit sentence matched the sentence alone (1/4), with the one surviving
line now reading `COMMENT`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@max-sixty
max-sixty merged commit ff99860 into main Sep 21, 2026
12 checks passed
@max-sixty
max-sixty deleted the comment-verdct branch September 21, 2026 19:24
@max-sixty max-sixty mentioned this pull request Sep 22, 2026
max-sixty added a commit that referenced this pull request Sep 22, 2026
Bumps the generator to 0.2.15, syncs `uv.lock`, and adds the 0.2.15
changelog section. The release ships the restrict-updates ruleset
requirement and the `AGENTS.override.md` pin (#1367), the follow-up
review that focuses on the pushed commits (#1368), a rising bar for
further fix rounds on PRs with no human author (#1366), #1365, #1362,
and the setup-uv v10.2.0 bump (#1359, #1360).

`uv run pytest` (1216 passed), the worker's vitest suite and typecheck,
and every pre-commit hook pass on this commit.

> _This was written by Claude Code on behalf of max-sixty_

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
max-sixty pushed a commit to max-sixty/leaf that referenced this pull request Sep 22, 2026
Regenerates the nine `tend-*` workflows on tend 0.2.15, up from 0.2.14.
Every job now pins `max-sixty/tend/claude@0.2.15`, so merging adopts
that release's agent behaviour as well as the YAML below.

## What changes for this repo

**`tend check` holds branch protection to a ruleset the bot cannot
bypass.** Required reviews alone no longer pass the
`branch-protection:main` check: the bot holds write, so its own approval
satisfies a review requirement on a pull request someone else opened,
and it can then merge. A read it cannot decode is now a failed read
rather than a pass — previously an unreadable protection response
reported the check green without having verified anything. Leaf already
carries the `Merge access` restrict-updates ruleset, so this check stays
`PASS`; `tend check --fix` now offers to create that ruleset for any
failing branch, not only for the specific "bot can still merge" message.

**`tend-mention` measures its queue delay before checkout and
`setup:`.** The delay step moved ahead of both, so the number it reports
is the wait for the job to start rather than that wait plus however long
this repo's `setup:` takes. Leaf's mention runs will report a smaller
delay than before for the same actual queue time.

**Reviews open with a finding.** A review body no longer spends its
first lines restating where GitHub shows the verdict, a follow-up review
focuses on what was pushed while keeping the whole PR in view, and each
further fix round on a PR with no human author has to clear a higher bar
before the bot pushes again.

**Action pins.** `astral-sh/setup-uv` moves from v10.1.0 to v10.2.0 in
the two workflows that set it up directly (`tend-mention`,
`tend-notifications`).

[Compare
0.2.14...0.2.15](max-sixty/tend@0.2.14...0.2.15)

<details><summary>Upstream commits in this range</summary>

- `chore: regenerate workflows with tend 0.2.14` (max-sixty/tend#1358)
- `fix(review): say where GitHub shows a review's verdict, so the body
opens with a finding` (max-sixty/tend#1362)
- `fix(mention): compute the queue delay before checkout and setup`
(max-sixty/tend#1365)
- `fix(review): raise the bar for each further fix round on a PR with no
human author` (max-sixty/tend#1366)
- `Require a restrict-updates ruleset, pin AGENTS.override.md, and
rewrite the security docs` (max-sixty/tend#1367)
- `chore: bump setup-uv to v10.2.0` (max-sixty/tend#1359)
- `chore: bump repo-local setup-uv and wrangler pins`
(max-sixty/tend#1360)
- `chore(token-usage): drop the stream-json artifact copy its own
comment marked for removal` (max-sixty/tend#1361)
- `fix(review): focus a follow-up review on what was pushed, with the
whole PR in view` (max-sixty/tend#1368)
- `chore: release 0.2.15` (max-sixty/tend#1369)

The `AGENTS.override.md` pin in max-sixty/tend#1367 and the dropped
stream-json artifact copy in max-sixty/tend#1361 have no effect here —
the first applies to the Codex harness and the second to tend's own
repository.

</details>

Co-authored-by: leaf-agent <318509791+leaf-agent@users.noreply.github.com>
max-sixty pushed a commit to max-sixty/worktrunk that referenced this pull request Sep 22, 2026
Regenerates the nine `tend-*.yaml` workflows with tend 0.2.15 (from
0.2.14). The version bump and the `astral-sh/setup-uv` pin (v10.1.0 →
v10.2.0, in the `tend-mention` and `tend-notifications` preflight jobs)
are mechanical; one step actually moves, in `tend-mention.yaml`.

**`tend-mention` measures its queue delay before checkout and setup.**
The "Compute queue delay" step used to run after `actions/checkout` and
`./.github/actions/tend-setup`, so the number it recorded was the wait
for the job to start *plus* the several minutes this repo's setup action
takes to install the Rust toolchain and sandbox tools. It now runs
first, and the delay reports the queue wait alone
([max-sixty/tend#1365](max-sixty/tend#1365)).

Behavior that arrives inside the action rather than in these files, and
that shows up in what the bot posts here:

- **PR reviews lead with a finding.** A review body now opens with
something actionable instead of restating the verdict GitHub already
renders in its own header
([max-sixty/tend#1362](max-sixty/tend#1362)),
and a follow-up review scopes itself to what the latest push changed
while still reading the whole PR
([max-sixty/tend#1368](max-sixty/tend#1368)).
- **Fewer self-directed fix rounds on bot PRs.** On a PR with no human
author the review session applies its own findings; each further round
now has to clear a higher bar, so a bot PR stops accumulating marginal
follow-up commits
([max-sixty/tend#1366](max-sixty/tend#1366)).
- **`AGENTS.override.md` joins the pinned instruction paths.** Codex
reads it in preference to `AGENTS.md`, so on a PR-derived checkout it is
now restored from the base commit alongside `CLAUDE.md`, `AGENTS.md`,
`.claude/`, and `.agents/` — a fork can no longer reach the session
through that filename. The same release makes a restrict-updates ruleset
a `tend check` requirement; this repo already satisfies it (`tend check`
passes every line today)
([max-sixty/tend#1367](max-sixty/tend#1367)).

Full comparison:
[0.2.14...0.2.15](max-sixty/tend@0.2.14...0.2.15).

Co-authored-by: worktrunk-bot <254187624+worktrunk-bot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants