Skip to content

fix(copilot): honor reasoning effort and deliver pinned review diffs - #567

Open
jcarosi wants to merge 4 commits into
gmickel:mainfrom
jcarosi:fix/copilot-high-effort-upstream-20261018
Open

jcarosi wants to merge 4 commits into
gmickel:mainfrom
jcarosi:fix/copilot-high-effort-upstream-20261018

Conversation

@jcarosi

@jcarosi jcarosi commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Why

Honor the selected Copilot review effort for Claude and OpenAI models, and give shell-less reviewers the exact pinned patch.

What changes for a user or operator

Claude review selections such as copilot:claude-opus-5.5:high now send --reasoning-effort high. Implementation, plan and completion reviews, including each implementation fan-out draw, receive the full pinned diff at a readable path.

Scope

14 files changed; +555/-97 lines; 3 generated, 2 mechanical.

1. Make High effort and patch coverage real

The adapter previously dropped Claude-family effort. Shell-less review sessions also lacked a readable pinned patch.

2. Forward effort and materialize pinned diffs before dispatch

Send the resolved effort on non-floor Copilot runs. Reuse the existing guarded diff writer before managed or CLI execution, with per-receipt/range/axis identity and failure before dispatch.

1 generated file

3. Verify argv, fallback receipts and diff transport offline

Tests assert the CLI option/value, precedence, floor behavior, byte-equal pinned diffs, fan-out identity and failure before managed/CLI execution. Both affected diff-boundary tests now run POSIX argv and Windows stdin subtests on every host; the implementation dispatch test also initializes its hooks independently.

4. Document the operator contract and retain review provenance

Explain readable patch delivery under existing tool denials. Keep historical attempts at their original ranges and record the fresh completion verdict.

Rest of diff: 2 mechanical files; 2 generated files

Coverage: R1 → groups 1, 2, 3, 4; R2 → groups 1, 2, 3, 4; R3 → groups 1, 2, 3, 4; R4 → groups 1, 2, 3, 4

Blast radius

Copilot review argv, effort attribution, diff transport and their tests/docs. Existing low/medium/high/xhigh values, configuration precedence, session handling and read-only restrictions remain supported. The auto floor omits the effort option; it does not attribute an effort that was not selected.

Verification

  • Focused regression suites: 345 tests passed: backend spec 109, model resolution 64, review convergence 172.
  • Full offline suite: Before the Windows test correction: 4061 tests, 0 failures/errors, 7 skipped across 218 files; hosted CI verifies the final head.
  • Static/generated checks: Ruff 0.16.0, manifest, Codex mirror, documentation anchors, spec validation and git diff --check pass.
  • Implementation review: Historical complete fan-out 5388114c: SHIP on 627efd0..97f0cd4a; the port retains identical runtime source hunks.
  • Fresh completion review: Official Copilot SHIP on f360d3a..ae80c9c; R1-R4 and G1/G2/G4 covered; 0 introduced findings.
  • Effective reviewer provenance: Native session.start: claude-opus-5.5, reasoningEffort high, Copilot CLI 1.0.98; pinned code/test diff read in chunks.
  • Real-world use: Earlier 8.8.0 fix supported Opus 5.5 High reviews during XPosed delivery; its PR feat: add Beads integration to flow skills #2 is merged with green CI.
  • Windows test correction: 958ea8b reproduces and fixes the initial Windows StopIteration errors; 64 focused tests pass with both transport channels.
  • Scoped correction reviews: Official SHIP on 02467aa..958ea8b and 958ea8b..88a06d0; introduced P3 corrected; no remaining introduced findings.
  • Hosted CI correction: The initial run exposed two Windows StopIteration errors; both are fixed and the final full Windows suite passes.
  • fail: Legacy Flow validation: 23 pre-existing fn-16/fn-24 heading errors, identical to upstream; contribution spec validates individually.
  • Final hosted CI: 13/13 checks pass at 88a06d0: https://github.com/gmickel/flow-next/actions/runs/39080275454
  • Hosted full-suite results: 4061 tests per OS, 0 failures/errors; skipped: Linux 8, macOS 9, Windows 200. All three unit jobs pass.

Tradeoffs

Ported onto upstream 9.0.0 (f360d3a) without changing runtime source hunks from historical fan-out 5388114c (627efd0..97f0cd4a). The older diff-less SHIP is retained as history and excluded from accepted evidence. A fresh completion SHIP covers f360d3a..ae80c9c. The later test-only commits have scoped SHIP reviews on 02467aa..958ea8b and 958ea8b..88a06d0, including independent diff creation per transport. There is no UI QA surface.

This is independent of #546. Both touch flowctl.py, test_model_resolution.py and runtime docs, but no changed function overlaps. A combined merge preserves both behaviors and merges code/tests/docs automatically; only CHANGELOG and the generated manifest need mechanical reconciliation if both are integrated. Neither contribution includes the other's fix.

The earlier corrected 8.8.0 implementation was used for Opus 5.5 High reviews during the completed XPosed delivery (https://github.com/jcarosi/xposed/pull/2). Current 9.0.0 compatibility is verified here with deterministic tests and a live official completion review; XPosed was not changed or retested.

Open items

The pre-existing optional Copilot triage judge still has its older effort handling (P3); it is outside the review dispatch fixed here. CLI compatibility was exercised with Copilot 1.0.98. Maintainer-owned website follow-up is separate. Global Flow validation has the same 23 legacy heading errors as upstream; this spec validates cleanly. Gordon retains the upstream merge decision.

Port the preserved b5d103c1/97f0cd4a/d216d39c contribution onto upstream 9.0.0 f360d3a. Python and test hunks match the previously reviewed source exactly; retain historical Flow review attempts at their original ranges and add the Unreleased entry. This contribution does not include or depend on PR gmickel#546. Validation: 4061 tests, 0 failures, 0 errors, 7 skips; 345 focused tests; Ruff, manifest, mirror, anchors and scoped Git checks pass.
Official Copilot completion review returned SHIP on f360d3a..ae80c9c. Native session 5bb1c8da-f36a-45ee-82de-10cb68c32974 records claude-opus-5.5, high, and CLI 1.0.98; the reviewer read the complete code/test diff in chunks and verified R1-R4 plus G1/G2/G4. Historical implementation SHIP remains bound to 627efd0..97f0cd4a; source/test hunks are unchanged by the upstream port. The pre-existing optional triage-judge P3 remains out of scope.
Copilot AI balanced review requested due to automatic review settings October 17, 2026 23:28

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two new tests assume POSIX prompt delivery and raise StopIteration on Windows.

1 open issue
What changed in this PR

Updates Flow-Next’s Copilot review backend to honor selected reasoning effort and provide shell-less reviewers with pinned diffs.

Changes:

  • Forward reasoning effort across model families while preserving fallback-floor behavior.
  • Materialize pinned diffs before primary and fan-out review dispatch.
  • Update regression tests, documentation, and review records.
File Description
plugins/​flow-next/​tests/​test_review_convergence_cap.py Updates effort-attribution assertions.
plugins/​flow-next/​tests/​test_model_resolution.py Tests diff delivery, fan-out identity, and dispatch failures.
plugins/​flow-next/​tests/​test_backend_spec.py Tests reasoning-effort arguments and precedence.
plugins/​flow-next/​skills/​flow-next-impl-review/​workflow-cli.md Documents Copilot diff access.
plugins/​flow-next/​scripts/​flowctl.py Implements effort forwarding and pinned-diff delivery.
plugins/​flow-next/​scripts/​flowctl_tracker/​MANIFEST.json Refreshes the source checksum.
plugins/​flow-next/​docs/​flowctl.md Documents the diff-delivery contract.
plugins/​flow-next/​codex/​skills/​flow-next-impl-review/​workflow-cli.md Synchronizes generated workflow guidance.
plugins/​flow-next/​codex/​docs/​flow-next/​flowctl.md Synchronizes generated runtime documentation.
CHANGELOG.md Records the user-facing correction.
.flow/​tasks/​fn-304-honor-copilot-reasoning-effort-for.1.md Preserves implementation and validation evidence.
.flow/​tasks/​fn-304-honor-copilot-reasoning-effort-for.1.json Records completed task state.
.flow/​specs/​fn-304-honor-copilot-reasoning-effort-for.md Defines effort and diff-transport requirements.
.flow/​specs/​fn-304-honor-copilot-reasoning-effort-for.json Preserves review history and completion status.

Review effort: Balanced

Comment thread plugins/flow-next/tests/test_model_resolution.py Outdated
Fix the two hosted Windows StopIteration errors by explicitly exercising POSIX argv and Windows stdin transport. Preserve byte-equal pinned diff, range, receipt identity and tool-denial assertions; initialize hooks so the implementation dispatch test also runs independently. No runtime behavior changes. Validation: all 64 model-resolution tests and Ruff pass; native implementer provenance verified.
Address correctness:1 from official review e85b332f: remove the prior pinned diff before every POSIX and Windows subtest so the exists-before-CLI assertion proves fresh materialization independently. Only two setup lines change; all 64 model-resolution tests pass and native implementer provenance is verified.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants