Skip to content

fix(tests): follow command literals through forwarding helpers in the emit-command ratchet - #1018

Merged
christian-byrne merged 3 commits into
mainfrom
fix/emit-command-scanner-follows-forwarders
Oct 17, 2026
Merged

christian-byrne merged 3 commits into
mainfrom
fix/emit-command-scanner-follows-forwarders

Conversation

@christian-byrne

Copy link
Copy Markdown
Contributor

The blind spot

tests/comfy_cli/output/test_discovery.py::test_every_emitted_command_registers_a_schema is the ratchet that stops a command from emitting an envelope/1 without a schema registered in COMMAND_SCHEMAS. Its collector recorded a command= value only when it was an ast.Constant at the .emit call itself.

Every structured workflow edit routes through a local forwarder:

# comfy_cli/command/workflow_edit.py
def _finish(renderer, p, workflow, op, base_version, stdout, command: str) -> None:
    ...
    renderer.emit(payload, command=command, changed=not stdout)   # <- ast.Name, not ast.Constant

The literal lives one frame up, at the call site (_finish(..., "workflow clear")). So the scanner saw a Name, recorded nothing, and the ratchet never had an opinion about any of it. The guard has been blind for its entire existence.

Blast radius

The forwarder pattern is not confined to _finish. A repo-wide scan for "takes a command parameter and forwards that same name into .emit(command=...)" finds five forwarders across four files:

Forwarder File
_finish comfy_cli/command/workflow_edit.py
_emit_op comfy_cli/command/workflow_edit.py
_emit_terminal comfy_cli/command/jobs.py
emit_selected comfy_cli/selector.py
validate_api_workflow comfy_cli/command/workflow.py

Together they hid 10 command literals from the ratchet: jobs watch, validate, workflow validate, workflow add-node, workflow clear, workflow connect, workflow insert-workflow, workflow reset-doc, workflow set-node-field, workflow set-widget.

The two gaps it hid

Eight of those ten happened to be registered already. Two were not, and were not in LEGACY_UNREGISTERED either (that allowlist contains no workflow entries at all):

  • comfy workflow clear
  • comfy workflow reset-doc

Both are registered here as "workflow", which is what every other _finish-based command maps to — they emit the identical _finish op envelope (workflow / op / base_version / version / wrote) as the already-registered comfy workflow delete. Per the test's own docstring they are registered rather than allowlisted; LEGACY_UNREGISTERED does not grow.

The fix

Two static passes, in the test file:

  1. Find functions in comfy_cli/ that take a command parameter and forward that same name into .emit(command=...). Record the parameter's positional index and any literal default.
  2. For every bare-name call to a known forwarder, extract the string literal passed in that position — by keyword or positionally — and record it exactly as the existing scanner records a direct literal.

Deliberately conservative, because a ratchet that guesses is worse than one that abstains:

  • still a pure static scan; nothing imports or executes comfy_cli
  • a non-literal argument (a variable, an f-string) is skipped, not guessed
  • forwarders are matched by function name and followed only through bare-name calls, so an unrelated obj.emit_selected(...) is never assumed to be the same function
  • two same-named helpers with different signatures are marked ambiguous and dropped
  • *args / **kwargs splats suppress positional extraction and the default fallback

How the positive control proves it

A guard nobody has watched fail is not known to work, so this PR demonstrates the failure both ways.

Red — scanner fix applied, the workflow reset-doc registration reverted:

E       AssertionError: Commands emitting envelopes without a registered schema:
E         {'workflow clear': ['comfy_cli/command/workflow_edit.py'],
E          'workflow reset-doc': ['comfy_cli/command/workflow_edit.py']}
E         Add each to COMMAND_SCHEMAS in comfy_cli/discovery.py.
1 failed, 19 deselected

Before the scanner change that same state was green — which is the defect in one line.

Green — both halves applied: 20 passed.

Six new tests encode it so it cannot rot back:

  • four unit tests of the collector over synthetic source: a forwarded positional literal is found, a forwarded keyword literal is found, a literal signature default is used when the call omits the argument, and a non-literal argument yields nothing
  • one asserting a helper that takes command but never forwards it into .emit is not treated as a forwarder
  • test_workflow_edit_commands_are_reachable_only_through_the_forwarder, a ratchet over the real source tree listing seven commands reachable only via _finish/_emit_op. If the forwarder pass ever regresses, this fails loudly before the registration check can go quiet again.

Testing

  • tests/comfy_cli/output/test_discovery.py — 20 passed
  • full tests/ suite — 15 failed, 10102 passed, 33 skipped
  • baseline on untouched main — 15 failed, 10096 passed, 33 skipped

Zero failure delta; +6 passing tests. The 15 failures are pre-existing on main and untouched by this change (code_search, logs, node_deps, agent/test_command, build_digest_cache, and two test_file_utils umask assertions sensitive to the runner's umask).

ruff check and ruff format --check both clean.

Note

workflow disconnect is a ninth _finish call site arriving in #923 and is deliberately untouched here. Once both land, the strengthened scanner covers it automatically — and would have caught it had it gone unregistered.

`test_every_emitted_command_registers_a_schema` only recorded a
`command=` value when it was an `ast.Constant` at the `.emit` call
itself. Every structured workflow edit passes its command name to
`workflow_edit._finish`, which emits `command=command` — an `ast.Name` —
so the literal sits one frame up at the call site and the ratchet saw
nothing. The guard has been blind for its whole existence.

Teach the collector to follow a literal through a forwarder: pass 1
finds functions in `comfy_cli/` that take a `command` parameter and
forward that same name into `.emit(command=...)`; pass 2 extracts the
string literal each call site passes in that position, by keyword or
positionally, falling back to a literal signature default. It stays a
static scan — nothing imports or executes `comfy_cli` — and stays
conservative: a non-literal argument is skipped rather than guessed,
and forwarders are followed only through bare-name calls.

Also register the two commands the blind spot was hiding.
@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Team
  • Run ID: 4c93ecd3-eb2c-4169-9b83-066e45b20653
📥 Commits

Reviewing files that changed from the base of the PR and between 865bd99 and ae605a9.

📒 Files selected for processing: 3
  • comfy_cli/command/workflow_edit.py
  • tests/comfy_cli/command/test_workflow_edit.py
  • tests/comfy_cli/output/test_discovery.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 3 remain after this review.

📝 Walkthrough

The command scanner now discovers literal values passed directly to .emit and through eligible one-hop helper functions. The workflow schema map includes additional commands, and workflow delete-node emits a matching command name.

Changes: Command discovery

Layer / File(s) Summary
Static command scanning
tests/comfy_cli/output/test_discovery.py
The scanner parses Python sources and identifies helpers that forward a command parameter to .emit. It detects direct and forwarded literals, including positional, keyword, and default arguments. Tests cover unresolved values, ambiguous bindings, imports, and known command forwarders.
Workflow command registration and emission
comfy_cli/discovery.py, comfy_cli/command/workflow_edit.py, tests/comfy_cli/command/test_workflow_edit.py
COMMAND_SCHEMAS maps clear, reset-doc, and delete-node to the workflow schema. The delete-node operation emits workflow delete-node; tests check successful and missing-node results.

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to ae605

The command-discovery and workflow changes appear ready to merge after normal checks.

Pre-merge checks: Passed 2
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@coderabbitai
coderabbitai Bot requested a review from skishore23 October 16, 2026 23:50
@christian-byrne christian-byrne added the cursor-review Request Cursor bot review label Oct 17, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @christian-byrne.

Found 5 finding(s).

Severity Count
🟡 Medium 1
🟢 Low 3
⚪ Nit 1

Panel: 6/6 reviewers contributed findings.

Comment thread comfy_cli/discovery.py
Comment thread tests/comfy_cli/output/test_discovery.py Outdated
Comment thread tests/comfy_cli/output/test_discovery.py Outdated
Comment thread tests/comfy_cli/output/test_discovery.py Outdated
Comment thread tests/comfy_cli/output/test_discovery.py Outdated
`delete-node` set `renderer.command = "workflow delete-node"` but passed
`"workflow delete"` to `_finish`, so the SUCCESS envelope named a
different, real command — the cloud/local CRUD `comfy workflow delete`
(workflow.py) — while the error path fell back to the correct name.

The registration ratchet could not catch it twice over: it could not see
the literal at all (it is forwarded through `_finish`), and even once it
could, `comfy workflow delete` is itself registered, so the check passes
on a name that only looks right. Agents reading the success envelope were
routed to the wrong command and schema.

`comfy workflow delete-node` is already in COMMAND_SCHEMAS, so correcting
the literal resolves to the existing `workflow` schema with no new entry.
Pinned by a test asserting the success and error envelopes agree.
Review follow-ups on the forwarder pass.

Key forwarders by `(defining module, function name)` and resolve a call
site only when the forwarder is defined in that module or explicitly
imported into it. Bare-name keying failed open in both directions: two
same-named forwarders marked each other ambiguous and silently dropped
every call site of both, and a same-named non-forwarder had the
forwarder's default attributed to it.

Stop using `ast.walk` to decide whether a function forwards — it
descends into nested functions, lambdas and classes, so an outer
function was marked a forwarder when an inner closure had its own
`command`. Walk only the function's own body, and skip any function that
rebinds `command` before the emit, since a transformed value is not the
caller's literal.

Narrow the starred-argument guard: a positional splat cannot bind a
keyword-only `command`, and one appearing after `command`'s index does
not affect that index. Only skip when a splat sits at or before it.

Add a ratchet over all five known forwarders. Only `_finish`/`_emit_op`
have forwarder-only commands to assert on, so `_emit_terminal`,
`emit_selected` and `validate_api_workflow` could otherwise stop
resolving with nothing failing.

Narrow the docstring to the one untransformed hop actually covered and
name the known gap: `deploy promote`/`deploy rollback` build their names
with an f-string two hops down and stay unchecked.
@comfy-greenlight-bot

comfy-greenlight-bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Swarmhost agentic review

The detailed evaluation is available to employees in the internal Slack review thread.

Evaluation budget remaining for this pull request: 14 automatic and 15 manual.

Updated by Swarmhost's agentic review process.

@comfy-greenlight-bot comfy-greenlight-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pull request at head commit ae605a9 has been reviewed and approved by Swarmhost's agentic review process.

The detailed evaluation is available to employees in the internal Slack review thread.

@christian-byrne
christian-byrne merged commit 7bff935 into main Oct 17, 2026
19 checks passed
@christian-byrne
christian-byrne deleted the fix/emit-command-scanner-follows-forwarders branch October 17, 2026 01:49
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cursor-review Request Cursor bot review greenlight

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants