Skip to content

fix(multiagent): add DISPATCHED status to prevent fan-in double-execution on resume (#3633) - #3724

Open
SuperMarioYL wants to merge 1 commit into
strands-agents:mainfrom
SuperMarioYL:fix/graph-fan-in-dispatched-double-exec-3633
Open

fix(multiagent): add DISPATCHED status to prevent fan-in double-execution on resume (#3633)#3724
SuperMarioYL wants to merge 1 commit into
strands-agents:mainfrom
SuperMarioYL:fix/graph-fan-in-dispatched-double-exec-3633

Conversation

@SuperMarioYL

Copy link
Copy Markdown
Contributor

Description

When a graph node is dispatched (asyncio.create_task) but the process crashes before the node's generator body sets Status.EXECUTING (e.g., Lambda timeout, container restart, os._exit), the node is left PENDING. On session resume, _compute_pending_sources seeds only from EXECUTING nodes — so a dispatched-but-not-started node with no completed ancestor is indistinguishable from a bypassed dead branch. Its fan-in children are marked ready off the completed sibling alone, and the fan-in is double-executed on resume.

This adds a DISPATCHED status to the Status enum, set at the dispatch site immediately before asyncio.create_task. _compute_pending_sources now seeds from both EXECUTING and DISPATCHED, closing the window between dispatch and generator-body start. This completes the partial fix from #3390, which added the EXECUTING seed but left the dispatched-but-not-started gap.

Changes:

  • base.py: Add DISPATCHED = "dispatched" to Status enum (additive, non-breaking)
  • graph.py: Set DISPATCHED at the dispatch site in _execute_nodes_parallel; update _compute_pending_sources to seed from EXECUTING and DISPATCHED
  • test_graph.py: Add TestResumeDispatchedSibling with two tests verifying the fan-in is not double-executed when a sibling was dispatched but not started

Related Issues

Fixes #3633

Documentation PR

No documentation changes needed — this is an internal status enum addition with no new public API surface.

Type of Change

Bug fix

Testing

  • Added TestResumeDispatchedSibling with two tests:

    • test_dispatched_sibling_excluded_from_resume_frontier: focused unit test that injects DISPATCHED state directly (right.execution_status = DISPATCHED) and asserts the fan-in join is excluded from next_nodes_to_execute while the dispatched sibling right is included. It pins the resume-frontier computation; on raw master it raises AttributeError on Status.DISPATCHED.
    • test_resume_runs_fan_in_once_for_dispatched_sibling: real-dispatch end-to-end reproduction — a fast-completing left entry point and a right entry point held in the dispatched window (dispatched via the real _execute_nodes_parallel path, blocked before _execute_node sets EXECUTING). The snapshot is captured from left's AfterNodeCallEvent, deserialized into a fresh graph, and resumed; join must run exactly once with both parents' outputs present.
  • Both tests are red on master via distinct, valid signals: the end-to-end test fails with an assertion failure (join lands in next_nodes_to_execute → double-execution on resume, because right stays PENDING and is treated as a bypassed dead branch), while the unit test raises AttributeError on Status.DISPATCHED. Both are green on the branch.

  • All 83 existing test_graph.py tests pass

  • All 181 broader multiagent tests pass (excluding a2a which has a pre-existing missing-dependency issue)

  • ruff check and ruff format --check pass

  • mypy passes with no issues

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@SuperMarioYL
SuperMarioYL requested a review from a team as a code owner August 9, 2026 21:02
@github-actions github-actions Bot added bug Something isn't working area-multiagent Multi-agent related area-async Related to asynchronous flows or multi-threading python Pull requests that update python code labels Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-async Related to asynchronous flows or multi-threading area-multiagent Multi-agent related bug Something isn't working python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Graph session-resume can silently double-execute a fan-in node: _compute_pending_sources treats only EXECUTING nodes as in-flight

1 participant