Skip to content

fix(agent): resume nested agent-as-tool interrupts across rehydration - #3675

Open
strandly-the-agent wants to merge 8 commits into
strands-agents:mainfrom
strandly-the-agent:agent-tasks/3076
Open

fix(agent): resume nested agent-as-tool interrupts across rehydration#3675
strandly-the-agent wants to merge 8 commits into
strands-agents:mainfrom
strandly-the-agent:agent-tasks/3076

Conversation

@strandly-the-agent

@strandly-the-agent strandly-the-agent commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Description

Resuming an interrupt raised inside a nested agent-as-tool only worked while the orchestrator and the sub-agent shared the same in-memory Interrupt. The resume path read the human's answer off the sub-agent's copy, which the tool executor had registered into the orchestrator by reference, so the two were literally the same object. Once either agent is rebuilt from storage — a stateless handler recreating every agent per request — they are two independent objects, and the answer written to the orchestrator's copy is invisible to the sub-agent's: the sub-agent re-raises the same interrupt forever and the approved tool never runs.

This implements the simplification agreed in #3008's thread, and supersedes the approach in that PR (no ToolInterruptEvent change, no invocation_state routing):

  • Answers travel as data. Sub-agent interrupts propagate upward as copies whose ids are namespaced with a prefix derived from the outer tool use id, so ids stay unique when several sub-agents are invoked in one turn and the sub-agent-local id is recoverable by stripping the prefix. The prefix opens with an SDK-owned marker (v1:agent_as_tool:) and percent-encodes the tool use id, because both ids are model-derived: without the encoding a tool use id containing the separator matches another call's ids, and without the marker a tool use id of exactly v1 matches every interrupt the orchestrator raised itself. On resume, the orchestrator's already-persisted _interrupt_state.context["responses"] are filtered by that prefix and mapped back to local ids.
  • An ephemeral sub-agent's interrupted turn is parked by the orchestrator. preserve_context=False sub-agents cannot have a session manager (_agent_as_tool.py rejects that combination), so their turn is stored as one keyed entry in the orchestrator's own interrupt context and freed once it has been reinstated. A preserve_context=True sub-agent owns its state and keeps its turn in its own session; if it has none, that is warned about when the interrupt parks — before a human is asked for a response that could not be applied — and the resume reports an actionable error rather than silently dropping the answer.
  • A parked turn that cannot be reinstated keeps its interrupt pending. The event loop clears an agent's whole interrupt record as soon as a turn ends, so a failed reinstate raises the interrupt again instead of failing the call: the turn stays parked behind a still-pending interrupt and the response can be applied on a later attempt. Only the interrupts the parked turn is still waiting on are re-raised — the orchestrator also holds ids the sub-agent has already finished with.
  • Parking an interrupt preserves interrupt-context keys the event loop does not own. This matches how Graph/Swarm already treat their own interrupt context.
                                                          8 files changed, 1177 insertions(+), 88 deletions(-)
strands-py/src/strands/agent/_agent_as_tool.py           +329  -18
strands-py/src/strands/event_loop/event_loop.py           +30   -3
strands-py/src/strands/agent/agent.py                      +6    0
strands-py/src/strands/interrupt.py                        +2   -1
strands-py/tests/strands/agent/test_agent_as_tool.py     +772  -66
strands-py/tests/strands/event_loop/test_event_loop.py    +29    0
site/.../concepts/interrupts.mdx                           +7    0
site/.../concepts/multi-agent/agents-as-tools.mdx          +2    0

The adapter grew more than the behaviour change alone needs: seven static helpers that each re-derived the orchestrator and the id prefix from invocation_state were consolidated into one _ParentCall object, per the review note about internal helpers.

Related Issues

Fixes #3076. Implements the design mkmeral settled on in #3008 and supersedes the snapshot-plumbing approach there. Co-authored with seanalbert, whose PR framed the problem and the stateless-Lambda use case (Co-authored-by: trailer is on the first commit).

Documentation PR

Included here rather than deferred. preserve_context decides who keeps a sub-agent's interrupted turn, and therefore whether a resume survives a restart, but it was documented purely as conversation history — and agents-as-tools.mdx is the only place customers see preserve_context=True, in a snippet with no session manager, which is the one configuration that cannot resume after a restart. interrupts.mdx documented multi-agent nesting for Swarm and Graph only, so agents-as-tools now has its own section covering id namespacing and where the turn lives. Interrupt.id is now documented as opaque, matching every sibling handle in the SDK.

Type of Change

Bug fix

Testing

Behaviour: the four agent-as-tool configurations, each across a process boundary

sub-agent config restart between turns outcome
preserve_context=False, no session manager yes resumes — turn reinstated from the orchestrator's interrupt context, tool runs once
preserve_context=True + its own FileSessionManager yes resumes — answer mapped as data; the sub-agent's own message log is untouched
preserve_context=True, no session manager no (same process) resumes in memory
preserve_context=True, no session manager yes does not resume — warns at park time, logs at ERROR, and returns a tool error that tells the model the action did not run

A and B are now both pinned by end-to-end tests with a real FileSessionManager and every agent rebuilt between turns; B is the configuration issue #3076 describes literally, and it fails at the merge base (verified: turn2 stop_reason=interrupt executions=[] on a10881c71, end_turn / ['prod-db'] at this head).

Gates — run directly; hatch is not available in my environment, and this sandbox kills any single shell command at ~60s, so the suite was run in four chunks rather than one invocation:

$ python -m pytest tests -q -n 4          # in 4 chunks covering every directory under tests/
920 + 1361 + 1399 + 1165 = 4845 passed, 0 failed

$ ruff format --check <6 changed py files>     6 files already formatted
$ ruff check src tests                         All checks passed!
$ mypy ./src                                   Success: no issues found in 234 source files

ruff format --check src tests across the whole tree reports 13 files would be reformatted — byte-identical at the merge base, so pre-existing and untouched by this PR.

What the new tests pin down, and that they fail without the fix. Every one of these was verified by reverting the specific hunk and watching the test fail:

test negative control
test_nested_interrupt_resumes_after_rehydration (#3076 regression) fails on main: assert [] == ['prod-db']
test_nested_interrupt_resumes_after_rehydration_with_a_sub_agent_session_manager fails at the merge base — loops instead of resuming
test_nested_interrupt_survives_a_parked_turn_that_fails_to_load fails when the re-raise is reverted to a terminal error result
test_nested_interrupt_that_reraises_twice_runs_each_confirmed_action_once fails when the re-raise is not filtered to what the parked turn awaits
test_stream_resume_reraises_the_interrupt_when_the_parked_turn_cannot_be_loaded same two mutations
test_stream_resume_reraises_only_the_interrupts_the_parked_turn_still_awaits fails on the unfiltered re-raise
test_namespaced_interrupt_ids_are_not_captured_by_a_tool_use_id_of_the_scheme_marker fails with _NAMESPACE_TAG = ""
test_namespaced_interrupt_ids_* (2) fail when the id escaping is reverted
test_stream_interrupt_warns_when_a_context_preserving_sub_agent_has_no_session_manager fails without the park-time warning
test_stream_interrupt_parks_a_turn_that_is_isolated_from_the_sub_agent fails without the deep copy
test_event_loop_cycle_interrupts_preserved_when_after_tools_hook_raises fails when the after-tools rescue call site replaces the whole context

Two tests were fixed rather than added, because they passed while the behaviour they named was untrue: test_stream_resume_keeps_stored_turn_when_it_cannot_be_loaded asserted an in-memory dict on a mock orchestrator and could not see that the event loop then cleared it (replaced by the re-raise test above), and test_stream_interrupt_resume_skips_state_reset constructed the tool after setting the sub-agent's messages, so its reset baseline equalled the expected result and a reset would have been invisible. Two tests for a removed one-line private accessor were deleted; the branch that used it is covered by the resume tests.

Also exercised by hand on this branch: partial answers, three-level nesting, two sub-agents whose local interrupt ids collide, a sub-agent with a SummarizingConversationManager and one with a stateful model (conversation_manager_state and model_state both intact after a real restart, against a genuinely fresh manager/model as the control), a Graph node whose agent has an interrupting agent-as-tool, and a check that nothing new reaches the caller.

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Review loop ledger

round pass findings outcome
1 independent correctness review 2 should-fix, 3 nits both fixed (b3dc2cd0); 2 nits applied, 1 declined
2 adversarial testing, 5 probes 1 routing bug, 1 doc gap fixed (40e08f22)
3 verification of the fix commits APPROVE, 2 notes + 1 nit applied (b097fc36)
4 six independent fresh-context passes (correctness · adversarial · test-quality · API/DevX · issue-alignment · LLM-context) 1 blocker, 5 should-fix all fixed or answered below (65d373030f0355)
5 adversarial + test-quality review of those fixes 1 blocker in the fix itself, 2 judgement calls, 2 nits blocker fixed (21fd240); rest below

Round 4 found what rounds 1–3 missed, and round 5 found a defect in round 4's own fix — a sub-agent that interrupts twice had its first, already-consumed interrupt id re-raised, which re-entered this very bug. Both are now fixed with tests that fail without them. Rounds 1–3 were dispatched by the same identity that wrote the code; rounds 4–5 were not given that assumption.

Declined, with reasons

  • Broad except Exception in the reinstate path. It guards deserializing persisted data possibly written by a different SDK version, where the failure modes are open-ended; narrowing it would let an unanticipated type escape into the adapter's generic handler, which reports a plain tool error at WARNING — exactly the silent loss of a human's approval this PR exists to prevent. It logs at ERROR and returns a specific message, and Graph/Swarm call _InterruptState.from_dict with no guard at all.
  • No issue link on tests written during development. Per the root AGENTS.md, only a regression test for a filed bug links its issue; the two regression tests for [BUG] Nested agent-as-tool interrupts don't resume across rehydration (stateless / distributed execution) #3076 do link it.
  • Stamping the tool name into a propagated interrupt's name. It would make two sub-agents' interrupts easier to tell apart in an approval UI, but name is caller-visible and matched on, so prefixing it risks breaking name-based routing to fix a legibility problem. The tool call is already identifiable from the id, and a structured field is the better answer — worth its own change.
  • A retry cap on re-raising. See the first residual: bounded retries end in discarding the human's answer, which is the destructive outcome this path exists to avoid.

Residual / known limitations

  • A permanently unreadable parked turn re-prompts indefinitely. If a reinstate can never succeed — e.g. a sub-agent's conversation-manager class changed while its turn was parked — every resume re-raises the interrupt, so the human is asked again each time and the only machine signal is the ERROR log. This is deliberate: the alternative is to give up after N attempts and discard an approval a human already gave, and a recoverable annoyance beats destroying the answer. An operator who fixes the underlying cause gets a working resume. Worth a maintainer's opinion.
  • Sharing one Agent instance as a tool across several orchestrators is a foot-gun: _reset_agent_state is unconditional, so one orchestrator's call can clear a turn another has parked. The parked orchestrator still resumes correctly; the other call fails with a confusing error. Documented on that method rather than fixed, because making the reset aware of a foreign parked turn is a behaviour change deserving its own review.
  • A load_snapshot that fails after validation leaves the sub-agent with some fields applied and others not. The parked turn is kept, the interrupt stays pending, and an ephemeral sub-agent is reset on its next fresh call, so it self-heals. It is not invoked in that state unless it is itself still activated from an earlier turn in this process; noted in the code.
  • Interrupt ids parked by an earlier revision of this branch use the older prefix and are not recognised after the marker change: the resume treats the call as fresh and re-runs the sub-agent's turn. That only affects state persisted by a revision of this branch, which has never shipped, so no migration is included.
  • Not automated: two sub-agent calls interrupting concurrently in one turn, three-level nesting, and a resume prompt mixing answers for a sub-agent's interrupts with the orchestrator's own. All three were exercised by hand and hold; they are the coverage I would add next.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Resuming an interrupt raised inside a nested agent-as-tool only worked while the
orchestrator and the sub-agent shared the same in-memory Interrupt object. The
resume path read the human's answer off the sub-agent's copy, which the executor
registered into the orchestrator by reference, so the two were the same object.
Once either agent is rebuilt from storage - a stateless handler recreating every
agent per request - they are two independent objects and the answer written to
the orchestrator's copy is invisible to the sub-agent's.

Resume now works as data. Sub-agent interrupts are propagated as copies whose ids
are namespaced by the outer toolUseId, so ids stay unique when several sub-agents
are invoked in one turn, and the sub-agent-local id is recoverable by stripping
the prefix. On resume the orchestrator's persisted interrupt responses are mapped
back to local ids and handed to the sub-agent.

An ephemeral sub-agent (preserve_context=False) cannot have a session manager, so
its interrupted turn is stored as one keyed entry in the orchestrator's interrupt
context and consumed on resume. A sub-agent used with preserve_context=True owns
its state and keeps its interrupted turn in its own session; if it has none, the
resume reports an actionable error instead of silently dropping the answer.

Parking an interrupt now preserves interrupt-context keys the event loop does not
own, matching how Graph and Swarm already treat their own interrupt context.

Co-authored-by: seanalbert <seanalbert@users.noreply.github.com>
@github-actions github-actions Bot added size/l python Pull requests that update python code area-hil Human in the loop and suspend/resume area-multiagent Multi-agent related bug Something isn't working labels Aug 6, 2026
An interrupted turn stored on the orchestrator was removed before it was
deserialized, so a load failure - a schema version the running SDK does not
accept, for instance - destroyed the only copy of the turn the human had already
answered. The stored turn is now dropped only once it has loaded, and the failed
tool result distinguishes a turn that could not be loaded from a sub-agent that
never had one.
@strandly-the-agent

Copy link
Copy Markdown
Contributor Author

Review loop — round 1. An independent fresh-context reviewer found two real bugs; both are fixed in b3dc2cd0, with a test that fails without the fix. A second, adversarial pass timed out without producing findings and has been re-dispatched with a tighter brief. Round 2 — a new reviewer, given round 1's findings — is running.

# finding status
1 _restore_continuation removed the stored turn before load_snapshot could fail, so a load failure — e.g. a schema version the running SDK does not accept yet — destroyed the only copy of the turn the human had already answered fixed: the entry is dropped only after it loads
2 The failed-resume message always blamed a missing session manager, even when the real cause was a failed deserialization of a stored turn fixed: the message branches on whether a stored turn exists
3 nit: two helpers that only read invocation_state were instance methods applied (@staticmethod)
4 nit: a docstring said ephemeral sub-agents "may not have a session manager" applied — construction rejects that combination
5 nit: broad except Exception in _restore_continuation not applied, reason below

After the fixes: 4838 passed (full Python unit suite), ruff check and mypy ./src clean, four-config matrix unchanged (A/B/C resume, D fails loudly).

The disputed nit, and what round 1 verified for itself

Why the broad except stays. It guards deserializing persisted data that may have been written by a different SDK version, where the failure modes are open-ended (SnapshotException, a missing key, a conversation-manager state mismatch). Narrowing it would let an unanticipated type escape into the adapter's generic handler, which reports a plain tool error at WARNING — exactly the silent loss of a human's approval this PR exists to prevent. Nothing is swallowed: it logs at ERROR and returns a specific message. Graph and Swarm call _InterruptState.from_dict with no guard at all (graph.py:1191, swarm.py:111), so this is strictly more defensive than the surrounding code. The round-1 reviewer agreed it was defensible.

Verified independently by round 1 (re-run by the reviewer rather than taken from me): ruff format --check and ruff check on the changed files, mypy ./src, and pytest tests/strands/agent/test_agent_as_tool.py tests/strands/event_loop -q → 210 passed. They also copied test_nested_interrupt_resumes_after_rehydration onto a main worktree and confirmed it fails there (assert [] == ['prod-db'] — the confirmed tool never re-executes) and passes on this branch, so the regression test is not vacuous. They traced the f"{tool_use_id}:{local_id}" scheme through _executor.py:255, where the orchestrator's _interrupt_state.interrupts is actually populated, and by hand across two and three levels of nesting; confirmed Graph/Swarm never replace their interrupt-context dict wholesale, which is what makes _park_interrupt_context the right shape; and confirmed direct tool calls never populate invocation_state["agent"] and forbid interrupts outright, so the new resume path cannot regress that call path.

Tool use ids and interrupt ids are both model-derived strings, so one call's raw
prefix could be a literal prefix of another call's namespaced ids - a tool use id
of "<other-id>:v1" matches every interrupt the other call raises, because hook
interrupt ids start with "v1:". The answer was then routed to the wrong sub-agent
with a mangled id, and the other sub-agent's genuine interrupt was lost: it failed
with "no interrupt found" and the turn ended as though the human had resolved it.

Percent-encoding the tool use id keeps the separator out of the prefix, so prefix
matching is exact while a tool use id that itself contains the separator still
round-trips.

Also documents on _reset_agent_state that sharing one Agent instance as a tool
across several orchestrators lets one call clear a turn another has parked.
@strandly-the-agent

Copy link
Copy Markdown
Contributor Author

Review loop — round 2 (adversarial). Five focused probes against the propagation/resume mechanism. One real routing bug, fixed in 40e08f22; three probes held; one exposed a foot-gun I documented rather than fixed.

🔴 Answers could be routed to the wrong sub-agent call. Namespacing used a raw f"{tool_use_id}:{local_id}", and matching used startswith plus a slice. Hook interrupt ids always start with v1:, so a model-supplied tool use id of the form "<other-call-id>:v1" makes one call's prefix a literal prefix of every interrupt the other call raises — no id collision needed. The reproduction answered only call B; call A matched B's answered id, stripped the wrong number of characters, and forwarded a mangled id, so A's sub-agent failed with no interrupt found, the orchestrator recorded that as a failed tool result, and the turn ended as though A's confirmation had been resolved — A's genuine, unanswered interrupt was gone. Fixed by percent-encoding the tool use id so the separator cannot appear in the prefix; a tool use id that itself contains the separator still round-trips. Negative control: reverting the escaping fails the new test, restoring it passes.

Held: no double execution (approve → re-interrupt → approve again runs each tool call exactly once, in-process and across a real restart); no wrong recipient in the ordinary case (answer A only → A runs once, B never runs and stays pending, then answering B runs it once); rehydration with a stateful sub-agent (SummarizingConversationManager state and a stateful model's model_state both come back intact after a restart, verified against a genuinely fresh manager/model as the control).

The foot-gun this surfaced, and why it is documented rather than fixed

Sharing one Agent instance as a tool across several orchestrators: _reset_agent_state is unconditional, so orchestrator #2's call clears the live turn orchestrator #1 has parked on an interrupt. No routing violation and no double execution — #1 still resumes correctly, because it reinstates the turn from its own interrupt record, which overwrites the damage — but #2's own call then fails with a low-level must resume from interrupt with list of interruptResponse's, and there is a window where the shared instance is inconsistent.

Fixing that properly means making the reset aware of a foreign parked turn, which is a behaviour change beyond this bug and would need its own review. I documented it on _reset_agent_state instead, telling callers to give each orchestrator its own sub-agent instance. Listed as residual in the PR body.

After both fixes: 4840 passed (full unit suite), ruff format --check / ruff check / mypy ./src clean, four-config matrix unchanged, and the partial-answer, three-deep-nesting and colliding-local-id scenarios all still behave. A third pass verifying the two fix commits is running; I will post its verdict and then take this out of draft.

…defensively

The round-trip test asserted only that a separator-bearing tool use id maps its
own answers back, which the unescaped scheme also satisfied, so it did not guard
the escaping; it now pins the encoded id. Removing a consumed turn uses pop so a
concurrent removal cannot raise, and the restore failure path records that a
partial load leaves the sub-agent mismatched until its next fresh call.
@strandly-the-agent

Copy link
Copy Markdown
Contributor Author

Review loop — round 3: APPROVE. A third fresh reviewer verified the two fix commits and cleared them; its two non-blocking notes and one test-quality catch are applied in b097fc36, so this is out of draft.

  • The escaping is provably collision-free. quote(x, safe='') never emits a raw : (it encodes % too), so for one call's prefix to match another's ids the comparison would have to land on a : inside an encoded id — impossible unless the ids are equal. The reviewer brute-forced adversarial pairs (empty string, unicode, %3A, a literal :, a%25, ids ending in :v1, emoji) through _namespace_interrupts with zero collisions, and separately confirmed the old raw scheme does collide on ("a", "a:v1") — the exact bug.
  • One of my three new tests wasn't pulling its weight. test_namespaced_interrupt_ids_round_trip_a_separator_bearing_tool_use_id passed even with the escaping reverted — a self round-trip works under both schemes. It now asserts the encoded id (ob%3Av1:…), and fails on revert like the other two.
  • Defensive pop instead of del when dropping a consumed turn, so a concurrent removal cannot raise — the reviewer traced it as unreachable today, but it's free insurance.
  • The partial-load window is now recorded in the code: if load_snapshot fails after validation, the sub-agent has some fields applied and others not. The stored turn is kept for retry, the caller reports the error without invoking the sub-agent, and an ephemeral sub-agent is reset on its next fresh call, so it self-heals. Listed as residual in the PR body.
  • Declined, with reason: the three new tests carry no issue link. Per the root AGENTS.md, a test written during development carries no issue reference — only a regression test for a filed bug links one, which is why test_nested_interrupt_resumes_after_rehydration links [BUG] Nested agent-as-tool interrupts don't resume across rehydration (stateless / distributed execution) #3076 and these do not.

At head b097fc36: 4840 passed (full unit suite, exit 0), ruff format --check4 files already formatted, ruff checkAll checks passed!, mypy ./srcSuccess: no issues found in 234 source files, four-config matrix unchanged (A/B/C resume, D fails loudly). Three independent passes ran, every finding is fixed or declined in writing, and residuals are listed in the body — ready for human review.

@strandly-the-agent
strandly-the-agent marked this pull request as ready for review August 6, 2026 12:46
@strandly-the-agent
strandly-the-agent requested a review from a team as a code owner August 6, 2026 12:46
@mkmeral

mkmeral commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@strandly-the-agent do an independent review, use subagents with fresh context to get them to review our changes. are we good?

mkmeral
mkmeral previously approved these changes Aug 6, 2026

@mkmeral mkmeral left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can clean up the code a bit more, especially internal helpers, but not a blocker for bugfix

@strandly-the-agent

Copy link
Copy Markdown
Contributor Author

Are we good? Not quite — one 🔴, five 🟡. Six fresh-context passes on b097fc36 found a blocker my three earlier rounds missed, in the default configuration: when a stored sub-agent turn fails to load, _restore_continuation deliberately keeps it "so a later attempt can still apply the human's answer" — but the tool then yields a terminal error result, so the round has no interrupts left, event_loop.py:906 calls deactivate(), and that clears context wholesale. The retained turn, the pending interrupt and the approval are all destroyed in the same cycle, and re-submitting the same interrupt id raises ValueError. 4 of 6 passes hit this independently; the LLMINFO pass, running live Bedrock, watched the model report "Perfect! ✅ The prod-db record has been successfully deleted" in 2 of 3 runs while the guarded tool never executed.

# sev where what passes
1 🔴 _agent_as_tool.py:483-497 + event_loop.py:906 retained continuation + human's approval destroyed by deactivate(); retry raises ValueError; model can narrate success 4/6
2 🟡 _agent_as_tool.py:445 config D's failure went from loud and still-pending to end_turn, approval consumed — a real change the residuals list doesn't mention 2/6
3 🟡 _agent_as_tool.py:307-321 toolUseId == "v1" → prefix "v1:" matches every SDK interrupt id (quote("v1") is a no-op); same threat model round 3 half-closed 2/6
4–6 🟡 docs point at the failing cell · two sub-agents' interrupts indistinguishable in an approval UI · config B (the issue's literal repro) + event_loop.py:884 untested see below

Blocking question for you, mkmeral — findings 1 and 2 both come down to one call you already touched in #3008: when a nested resume can't be applied, should it stay an error tool result (status quo — the caller sees stop_reason="end_turn"), or re-park the interrupt so the answer stays live and the caller can retry? Re-parking was scoped out in #3008 and it's the clean fix for finding 1, so I don't want to just do it. Everything else is mechanical and I can push it on your word.

I can't formally approve/request-changes my own PR, so this is a comment. Two findings are pre-existing (not this PR's) and I'd file them separately rather than block here — say the word.

🔴 1 — the retained continuation is destroyed by deactivate() (full evidence)

_agent_as_tool.py:480-497 keeps the stored turn on a load failure, with the rationale that "dropping it here would destroy the only copy of the turn the human already answered." True in isolation. But stream() (_agent_as_tool.py:194-208) then yields ToolResultEvent(status="error") rather than a ToolInterruptEvent, so this round's interrupts list is empty, so event_loop.py:906 reaches agent._interrupt_state.deactivate() — which at interrupt.py:62-70 does self.interrupts = {}; self.context = {}. I re-read all four of those sites myself; the mechanism is exactly as described.

End-to-end repro (real Agent + FileSessionManager, continuation corrupted to simulate the rolling-upgrade schema skew the comment itself names):

round1: orchestrator("do it")   -> stop_reason=interrupt, interrupt_id=I
[corrupt persisted continuation: schema_version 1.0 -> 0.0]
round2: resumed(APPROVE)        -> stop_reason=end_turn (!), tool result = "error"
        persisted after round2: {'interrupts': {}, 'context': {}, 'activated': False}
round3: resumed2(APPROVE again)  # "corruption fixed, retry"
        -> ValueError: Received interrupt responses but agent is not in interrupt state

So "leave it for a later attempt" buys exactly zero additional attempts in the single-pending-call case — the primary case this mechanism targets. Reachability: any preserve_context=False (the default) agent-as-tool whose continuation transiently fails to load; the trigger is the version skew the code comment names, i.e. a rolling deploy of exactly the stateless handler this PR exists for.

Second manifestation, same root cause: two concurrently-interrupted sub-agent calls where one resolves normally and the other can't — the resolvable one emptying interrupts triggers the same deactivate(), silently dropping the other's never-answered interrupt.

Three artifacts currently assert a guarantee that doesn't hold: the code comment at :483-487, test_stream_resume_keeps_stored_turn_when_it_cannot_be_loaded (asserts the in-memory dict on a MagicMock orchestrator, never crosses into the event loop — it passes while the behaviour it names is impossible), and the PR body's "kept for a retry… so this self-heals." All three need to change with the fix.

Structural fix (needs your call): on an unresumable restore, re-emit a ToolInterruptEvent for this call's already-namespaced interrupts, read back from parent._interrupt_state.interrupts filtered by the prefix. The round stays non-empty → the loop takes _stop_for_interrupts/_park_interrupt_context instead of deactivate() → interrupt and stored turn both survive, as the comment claims. Fixes the two-call variant for free.

Cheap fix, orthogonal, ships either way_unresumable_message (_agent_as_tool.py:418-429) tells the model what failed but never that the guarded action didn't run, nor what not to do. That's what let the live model claim success:

return (
    f"Agent '{self._tool_name}' did NOT run: its interrupted turn failed to load, so the human's "
    "approval was NOT applied. Do not report this action as completed or successful. Tell the user "
    "it failed and ask them to approve again before retrying. See the logged error for the cause."
)

The SDK's own precedent for a refused action is blunt and imperative — types/interrupt.py:42 (DENIED: {reason}), interventions/registry.py:129,139, and vended_plugins/steering/core/handler.py:118 ("You MUST follow this guidance immediately"). This message should read like those.

🟡 2 — config D now fails silently instead of loudly

Same script, both refs:

turn 2 stop_reason interrupts tool ran
merge base a10881c7 'interrupt' (re-prompts — bad, but self-announcing, operation still pending) new id no
head b097fc36 'end_turn' [] no

The PR body says config D "returns an actionable tool error rather than dropping the approval". The error result is real, but the approval itself is gone and not re-submittable — which is the part that gets dropped. The deferral of the deeper fix (your 2026-08-05 call: "that's a new feature") stands; what's missing is that the failure mode changed, and the residuals list doesn't say so.

Cheapest honest improvement — warn at park time, before the human is asked for an approval that can never be applied, instead of at resume time after it's lost (_stash_continuation, _agent_as_tool.py:445):

if self._preserve_context:
    if getattr(self._agent, "_session_manager", None) is None:
        logger.warning(
            "tool_name=<%s>, tool_use_id=<%s> | sub-agent uses preserve_context=True with no session "
            "manager: its interrupted turn is held only in memory and will not survive a restart",
            self._tool_name, tool_use_id,
        )
    return

Fires in config C too (where in-process resume works fine), hence warning not error. ~5 lines, no behaviour change, respects the deferral.

For the record, the API pass measured the deferred feature rather than speculating: moving the gate from not self._preserve_context to "the sub-agent has no session manager" is 2 lines, leaves cells A/B/C byte-identical and makes cell D resume across a real rebuild. It also rolls back a sub-agent that accumulated history between park and resume — which is exactly the multiagent-session-semantics question you said deserves its own design. Input for that follow-up, not a request here.

🟡 3 — toolUseId == "v1" still collides with the SDK's own interrupt sentinel

Round 3's percent-encoding closed the raw-separator hole, but every SDK interrupt id starts with the literal v1: (hooks/events.py:169,244,438, types/tools.py:162, _middleware/stages.py:134) and quote("v1", safe="") is a no-op — I checked: quote("v1", safe="") + ":"'v1:'. So _namespace_prefix("v1") prefix-matches every interrupt the orchestrator raised itself, at _agent_as_tool.py:368 and :391.

Reproduced: orchestrator confirms its own dangerous_action while a sub-agent also interrupts, human approves both → the orchestrator's own answer is stripped and forwarded down to the sub-agent, which raises KeyError (no interrupt found), the tool errors, and the sub-agent's approved dangerous_action("prod-db") never runs while the turn reports end_turn. Control with tooluse_worker_123: both actions execute.

Reachability: narrow — 🟡, not a blocker. No Bedrock/OpenAI/Anthropic id format and no direct tool call (tools/_caller.py:109) ever emits "v1"; it needs a custom/local provider, a hook rewriting tool_use, or a prompt-steered model. But it's the same threat model round 3 accepted when it added the encoding, with the one remaining reserved key — and the fix is one line, in a namespace the SDK controls rather than something derived from model text:

return f"v1:agent_as_tool:{quote(tool_use_id, safe='')}:"

That also fixes the cosmetic complaint that nested ids no longer start with v1: while every sibling id site does. Cost: 8 tests hard-code the f"{tool_use_id}:{local}" shape — they'd be format-agnostic if they built expectations via _namespace_prefix. Worth deciding before merge either way, since these ids are client-persisted.

🟡 4–6 — docs, approval-UI legibility, test gaps

4 · The one documented preserve_context=True example is the cell that stays broken. agents-as-tools.mdx:141-147 is the only place customers see preserve_context=True, and the snippet has no session manager — so the documented happy path is config D, which per finding 2 now fails more silently than before. I'd previously called docs a follow-up; that was the wrong call, and the reason isn't AGENTS.md:339 in the abstract, it's that the existing docs now point at a newly-quieter failure. Three cheap spots: the preserve_context docstring (agent.py:977-981, duplicated at _agent_as_tool.py:72-76), the mdx caution inline, and interrupts.mdx (which documents Graph/Swarm nesting in detail and never mentions agent-as-tool).

5 · Two sub-agents' interrupts are indistinguishable to a human or LLM approval UI. _namespace_interrupts (:338-342) correctly leaves name/reason alone — but that means the only legible fields carry no agent identity, and AgentResult.interrupts threads bare Interrupts with no tool_use attached. Two sub-agents sharing one hook class (a normal reuse pattern) produce byte-identical name/reason, separable only by parsing an opaque compound id that with a real model-generated tool use id (tooluse_z7iPwa8Xr1Ww1i0RHGjNRV) says nothing. One-line fix: name=f"{self._tool_name}: {interrupt.name}" (makes _namespace_interrupts an instance method).

6 · Test gaps, both mutation-verified. (a) Config B — preserve_context=True + the sub-agent's own FileSessionManager — is the literal text of #3076's repro and has no end-to-end test; only config A does. It works (verified independently across a real two-process restart), it's just not pinned by CI. (b) Reverting only the event_loop.py:884 _park_interrupt_context call site passes the whole suite: test_park_interrupt_context_keeps_keys_the_event_loop_does_not_own proves the helper, not that call site, and the pre-existing after-tools test only asserts loop-owned keys. Seed a foreign key before the hook raises and assert it survives.

Ledger, what held, appendix, and what I suppressed

Ran: head b097fc36 vs merge base a10881c71. Six fresh-context passes — correctness, adversarial (advanced, real multi-process restarts), test-quality (mutation-tested), API/DevX (advanced), issue-alignment (two-process repro), LLMINFO (live Bedrock anthropic.claude-haiku-4-5) — after a triage route (ACCEPT) and a context-build pass. Triage wanted to skip four of these on the grounds that the PR's own tests cover them; that's circular for an agent-authored PR, so I overrode it and fanned out wide. Every file:line I cite above I re-read myself at this SHA.

✅ 1239 tests re-run green on the touched suites (the body's 4840-pass full-suite figure independently corroborated in shape). ✅ All three negative controls re-run and confirmed: regression test fails on main (assert [] == ['prod-db']), reverting the escaping fails both namespacing tests, reverting the stored-turn fix fails its test. ✅ Round-3's rewrite of the round-trip test does now earn its keep.

Held under attack (worth knowing what's not broken): 3-level nesting across a real restart; two concurrent sub-agents with byte-identical local interrupt ids, partial answers, exactly-once execution each; two interrupts inside one sub-agent call; a Graph node whose agent has an interrupting agent-as-tool; continuation GC across turns (deactivate() prevents accumulation); percent-encoding injectivity for every plausible id; no double execution on approve→re-interrupt→approve across restarts; the preserve_context=True + own-session-manager escape hatch; _park_interrupt_context is necessary, not a drive-by (reverting it returns config A to a silent drop). Scope matches the body digit-for-digit; types/_events.py is untouched, as claimed; the Co-authored-by: seanalbert trailer is on 73077edb2.

Alignment: fixes #3076 as filed. The issue's literal step 2 says preserve_context=True, and three converging pieces of evidence — the issue's own bug-verify repro (which gave the sub-agent a FileSessionManager), "rebuild both from the session store", and your #3008 comment — make config B the intended reading. Config B and the reporter's actual stated use case (config A, ephemeral, no session manager) both resume. Config D is broken only under an assumption the reporter never stated, and is disclosed.

⚪ appendix, non-blocking: _InterruptState.to_dict() returns context by reference, so a stashed continuation aliases the live dict (interrupt.py:120-126; unreachable through any path the adversarial pass could build — one deepcopy would close it for free) · an ephemeral sub-agent's parked tool input now lands in the orchestrator's session store where it previously landed nowhere, stranded if the human never answers (not caller-visible, cleaned on resume — probably a docstring line) · a sub-agent answer is single-use where an orchestrator's own answer is durable (recoverable, contrived trigger) · Interrupt.id has no documented opacity statement while every sibling handle in the SDK has one (snapshot_session_manager.py:108, checkpoint.py:47, storage.py:77) · neither new logger.error logs the sub-agent's own .name · _park_interrupt_context's "foreign keys survive" contract is unenforced folklore — a Final set of loop-owned keys would make it greppable.

Pre-existing, not this PR's — I'd file these separately rather than block: a transient hook failure on the resume cycle consumes an approval identically (verified byte-identical on the merge base, so it's the interrupt system's general error semantics) · Graph/Swarm route interrupts by a stored id list and would collapse two children raising the same v1:before_tools:{uuid5(name)} id into one entry (graph.py:739-744) — a latent bug this PR's namespacing happens to avoid for agent-as-tool.

Suppressed on purpose (so you can check my precision): the broad except Exception in _restore_continuation — declined round 1, no pass brought new evidence · issue links on the three new tests — declined round 3, and the test-quality pass independently confirmed AGENTS.md/TESTING.md supports the decline · the shared-Agent-instance residual — already disclosed, no pass found it worse than admitted. The other residual ("partial load_snapshot… self-heals") is not suppressed: that claim is false, and it's folded into finding 1.

Non-blocking questions: does a contract-only change (no public signature, but Interrupt.id shape and preserve_context resume semantics both moved) want an api/needs-review label under API_BAR_RAISING.md's "customer-facing contracts" clause? And is Interrupt.id opacity a deliberate non-commitment or just never written down?

@mkmeral

mkmeral commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@strandly-the-agent fix and iterate, run review subagents to review your fixes and final pr. make sure everything is good

@mkmeral
mkmeral self-requested a review August 6, 2026 19:57
…ot be reinstated

A parked sub-agent turn that failed to load was deliberately left in place so a later
attempt could still apply the human's response, but the call then yielded a failed tool
result. With no interrupt left in the round the event loop ends the turn and clears the
agent's whole interrupt record, taking the parked turn, the pending interrupt and the
response with it - and a resubmission of the same interrupt id then raises. Raise the
interrupt again instead, so the turn stays parked behind it and the response can be
applied once the turn loads.

Two further defences on the same path:

- Namespaced interrupt ids now open with an SDK-owned marker. Every interrupt id the SDK
  generates opens with the v1: scheme marker and percent-encoding leaves "v1" untouched,
  so a tool use id of exactly "v1" produced a prefix matching every interrupt the
  orchestrator raised itself, handing its own responses down to a sub-agent.
- A context-preserving sub-agent with no session manager is warned about when its
  interrupt parks, rather than only when a response arrives that can no longer be applied.

The failed-resume tool result now also tells the model the guarded action did not run and
that it must not report success, which is what a model reads before telling the user.

The orchestrator's side of a call moves into a _ParentCall object that binds the
orchestrator to the call's id namespace, replacing seven static helpers that each
re-derived both from invocation_state.
Parking a turn refreshes the loop's own keys and carries everyone else's, which was three
string literals in a comprehension. Name them once so the rule is greppable, and cover the
after-tools rescue call site: reverting only that site left the whole suite green, because
the existing test asserts the loop's own keys and never a foreign one.
…eserve_context mode

preserve_context decides who keeps a sub-agent's interrupted turn, and so whether a resume
survives a restart, but it was documented purely as conversation history - and the one
place preserve_context=True is shown to customers has no session manager, which is the
configuration that cannot resume after a restart. Interrupts documented multi-agent
nesting for Swarm and Graph only, so agents-as-tools gets its own section covering id
namespacing and where the turn lives. Interrupt.id is now stated to be opaque, matching
every sibling handle in the SDK.
Comment on lines +41 to +79
def _namespace_prefix(tool_use_id: str) -> str:
"""Build the prefix that namespaces a sub-agent interrupt id to one agent-as-tool call.

Two sub-agents invoked in the same turn can raise the same interrupt id, because each derives it
from its own tool use id. The prefix keeps them distinct in the orchestrator's interrupt record and
keeps the sub-agent-local id recoverable by stripping it back off.

Tool use ids and interrupt ids are both model-derived, so the prefix defends against both ways one
call's prefix can match another id: the tool use id is percent-encoded, because otherwise one
containing the separator matches another call's ids, and the prefix opens with a reserved marker,
because otherwise a tool use id of ``v1`` matches every interrupt the orchestrator raised itself.

Args:
tool_use_id: Tool use ID of the agent-as-tool call.

Returns:
Prefix, separator included, for interrupt IDs belonging to that call.
"""
return f"{_NAMESPACE_TAG}{quote(tool_use_id, safe='')}:"


def _namespace_interrupts(tool_use_id: str, interrupts: list[Interrupt]) -> list[Interrupt]:
"""Copy sub-agent interrupts with their ids namespaced to one agent-as-tool call.

Only the id changes: ``name`` and ``reason`` are what a human or an approval UI reads, so they are
passed through untouched.

Args:
tool_use_id: Tool use ID of the agent-as-tool call.
interrupts: Interrupts raised inside the sub-agent.

Returns:
Orchestrator-visible copies carrying namespaced ids.
"""
prefix = _namespace_prefix(tool_use_id)
return [
Interrupt(id=f"{prefix}{interrupt.id}", name=interrupt.name, reason=interrupt.reason)
for interrupt in interrupts
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these really need to be their own functions?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

return self._tool_use_id

@property
def awaiting_resume(self) -> bool:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot of new terminology introduced: awaiting_ resume, parked and interrupted. Should we clarify these terms and consolidate where needed so we're consistent through the class and codebase?

Comment on lines +491 to +494
f"Agent '{self._tool_name}' did NOT run and the human's response was NOT applied: its "
"interrupted turn did not survive the restart. A sub-agent used with preserve_context=True "
"keeps its own state, so it needs its own session manager for that state to survive. Do not "
"report the requested action as completed or successful; tell the user it failed."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the orchestrator know if preserve_context=True was used by the sub agent?

finally:
self._lock.release()

def _reset_agent_state(self, tool_use_id: str) -> None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this also reset _interrupt_state?

…l awaits

The orchestrator keeps every interrupt id it has been handed until its own turn ends, so a
sub-agent that interrupts twice leaves its first id recorded there after that step has
already run. Re-raising everything with the call's prefix therefore offered the caller an
id the reinstated sub-agent no longer holds; answering it made resume raise, the call
failed, and the round ended with no interrupt left - so the parked turn and the answer were
cleared, which is the failure this path exists to prevent. Offer only what the parked turn
is still waiting on.

Also: park a copy of the turn rather than the snapshot's own dicts, which alias the
sub-agent's live interrupt context, and correct the claim that a sub-agent is never invoked
after a partly applied load - it is, if it is still activated from an earlier turn in this
process.
@strandly-the-agent

Copy link
Copy Markdown
Contributor Author

Fixed and iterated — and the review of my fixes found a blocker in the fix itself, which is now fixed too. Head is 21fd240. The re-park fix I wrote for round 4's blocker re-raised every interrupt id the orchestrator still held for the call, including ones the sub-agent had already finished with. So a sub-agent that interrupts twice got its first, already-executed id handed back; answering it made resume() raise, the call failed, the round ended with no interrupt — and the parked turn plus the answer were cleared. That is #3076's exact failure mode, re-entered through its own fix. It now re-raises only what the parked turn is still waiting on, verified against the reviewer's own repro.

# round-4 finding fix
1 🔴 parked turn + approval destroyed by deactivate(); retry raised ValueError re-raise the interrupt instead of failing the call, so the turn stays parked behind it (65d3730)
1b 🔴 new, found reviewing that fix — re-raise offered a consumed interrupt id filter to the interrupts the parked turn still awaits (21fd240)
2 🟡 config D failed silently, discovered only after a human answered warn at park time, before anyone is asked; error text now tells the model the action did NOT run and not to report success
3 🟡 toolUseId == "v1" captured the orchestrator's own interrupts prefix opens with an SDK-owned v1:agent_as_tool: marker
4 🟡 docs pointed at the one config that cannot resume preserve_context docstrings, the agents-as-tools caution, a new agents-as-tools section in interrupts.mdx, Interrupt.id documented opaque
6 🟡 config B (the issue's literal repro) and event_loop.py:884 untested e2e test for B (fails at the merge base), foreign-key assertion at that call site
your note: "clean up the code a bit more, especially internal helpers" seven static helpers that each re-derived the parent and prefix from invocation_state → one _ParentCall object

✅ Verified at 21fd240: 4845 passed / 0 failed · ruff check clean · mypy clean (234 files) · 11 negative controls run — every new test was watched failing with its fix reverted.

Two judgement calls I'd rather you make than me (both in the body's residuals):

  1. A permanently unreadable parked turn now re-prompts the human indefinitely instead of failing. I chose recoverable-but-annoying over destroying an approval, and declined a retry cap for that reason — but a livelock is a real cost, so tell me if you'd rather it gave up after N.
  2. Finding 5 from round 4 — stamping the tool name into a propagated interrupt's name so an approval UI can tell two sub-agents apart — I declined: name is caller-visible and matched on. A structured field is the right answer, as its own change.

I did not file the two pre-existing bugs as issues yet — say the word and I will.

The blocker in my own fix, in detail

The orchestrator keeps every interrupt id it has been handed until its own turn ends — nothing GCs a consumed one. So for a sub-agent with two guarded steps ("read the row, then delete it"), by the second interrupt the orchestrator holds [sub-1 (answered, already ran), sub-2 (pending)] while the parked turn holds only sub-2.

My pending_interrupts() returned both. The reviewer's repro, before the fix:

re-park round -> pending: ['sub-1', 'sub-2']     # sub-1 resurrected
final round: answered every interrupt handed back
   agent invocation failed: 'interrupt_id=<...sub-1...> | no interrupt found'
   stop_reason: end_turn | executions: ['first']   <- 'second' never ran
   parked turns left: {}                           <- and the answer is gone
   resubmit -> ValueError: ... agent is not in interrupt state

And after (same script, unmodified):

re-park round -> pending: ['sub-2']
final round: stop_reason: end_turn | executions: ['first', 'second']
   toolResult: success

The fix reads the parked snapshot's own interrupt_state.interrupts and intersects. Reachability was the reason this rated 🔴 rather than a nit: two guarded steps in one sub-agent is ordinary HITL, and a caller answering exactly what it was handed is the documented contract — the SDK's own example iterates result.interrupts.

A side effect worth noting: this makes the _unresumable_message "failed to load" branch reachable again. The reviewer had proved it was dead code under the unfiltered version; with the filter, a parked turn holding no matching interrupt now correctly falls through to the terminal error.

What the fix review ruled out, and what's still open

Held under attack (adversarial pass, real multi-process restarts): re-parking is idempotent and terminates — three re-park cycles replayed no tool, grew no messages, made no model calls and charged no tokens, with the unrelated tool in the same batch still running exactly once; two sub-agent calls where one is resumable and one corrupt — the answered one ran once, the corrupt one stayed pending; no silent auto-approval from the stale .response on a re-raised interrupt (the loop drops responses on every park, so an empty resume executes nothing, and a later DENY overrides an earlier APPROVE); config D still terminal, not looping; the _ParentCall refactor is behaviour-identical to the previous head, and the deepcopy fixes a real aliasing leak that was present before.

Test-quality pass: all four claimed negative controls reproduced independently, the merge-base failure for config B confirmed, and the vacuous test I'd flagged confirmed genuinely fixed. It found one gap I'd missed — the deepcopy was unexercised (removing it left all 50 tests green), so there's now a test that fails without it. It also confirmed deleting the two tests for the removed one-line accessor lost no coverage.

Still not automated (listed in the body): two sub-agent calls interrupting concurrently in one turn, three-level nesting, and a resume prompt mixing sub-agent and orchestrator answers. All three pass by hand; the concurrency one is the most worth adding, since namespacing exists precisely for it.

Deliberately not done: no migration for interrupt ids parked by an earlier revision of this branch under the old prefix — that format never shipped. The failure shape there is a silent replay of the sub-agent's turn rather than a loud reject; if you'd rather it were loud pre-merge, that's a one-line warning.

@yonib05 yonib05 added complexity/high A touched function exceeds cognitive complexity 25; may be worth splitting size/m and removed size/l labels Aug 7, 2026
Comment on lines +41 to +79
def _namespace_prefix(tool_use_id: str) -> str:
"""Build the prefix that namespaces a sub-agent interrupt id to one agent-as-tool call.

Two sub-agents invoked in the same turn can raise the same interrupt id, because each derives it
from its own tool use id. The prefix keeps them distinct in the orchestrator's interrupt record and
keeps the sub-agent-local id recoverable by stripping it back off.

Tool use ids and interrupt ids are both model-derived, so the prefix defends against both ways one
call's prefix can match another id: the tool use id is percent-encoded, because otherwise one
containing the separator matches another call's ids, and the prefix opens with a reserved marker,
because otherwise a tool use id of ``v1`` matches every interrupt the orchestrator raised itself.

Args:
tool_use_id: Tool use ID of the agent-as-tool call.

Returns:
Prefix, separator included, for interrupt IDs belonging to that call.
"""
return f"{_NAMESPACE_TAG}{quote(tool_use_id, safe='')}:"


def _namespace_interrupts(tool_use_id: str, interrupts: list[Interrupt]) -> list[Interrupt]:
"""Copy sub-agent interrupts with their ids namespaced to one agent-as-tool call.

Only the id changes: ``name`` and ``reason`` are what a human or an approval UI reads, so they are
passed through untouched.

Args:
tool_use_id: Tool use ID of the agent-as-tool call.
interrupts: Interrupts raised inside the sub-agent.

Returns:
Orchestrator-visible copies carrying namespaced ids.
"""
prefix = _namespace_prefix(tool_use_id)
return [
Interrupt(id=f"{prefix}{interrupt.id}", name=interrupt.name, reason=interrupt.reason)
for interrupt in interrupts
]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

prompt = str(tool_input)

tool_use_id = tool_use["toolUseId"]
parent_call = _ParentCall.resolve(invocation_state, tool_use_id)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just general question on code readability: _ParentCall.resolve returns an object if invocation_state.get("agent") is not None, else it returns none. If that logic lived inline here, I think it would be easier for a human to read and follow the code since there are less jumps/redirects. This code is valid and fine, but curious if we should try to index on "human readability" of code in these reviews?

An agent exposed as a tool with [Agents as Tools](./multi-agent/agents-as-tools.md) propagates its interrupts up to the orchestrator, so the orchestrator's caller sees them and answers them like any other interrupt. Two things are worth knowing:

- **Interrupt ids are namespaced by the tool call.** A sub-agent's interrupt arrives with an id derived from the orchestrator's tool call, so two sub-agents interrupting in the same turn stay distinct. Treat the id as opaque: pass it back unchanged to resume, and use `name` and `reason` for anything you show a human.
- **Where the interrupted turn is kept depends on `preserve_context`.** With the default `preserve_context=False` the orchestrator carries the sub-agent's interrupted turn, so the resume survives the orchestrator and sub-agent being rebuilt from storage. With `preserve_context=True` the sub-agent owns its own state and needs its own session manager for the resume to survive a restart.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment makes sense in context to this pr, but reads as a bit out of place in this doc page. If im reading this, I dont really care where the interrupt is kept, I just want to know that I can interrupt my agent as a tool. Im worried that including this might just confuse people

</Tab>
</Tabs>

Interrupts raised inside a sub-agent resume automatically, but where the interrupted turn is kept depends on this setting. With the default <Syntax py="preserve_context=False" ts="preserveContext: false" />, the orchestrator carries the sub-agent's interrupted turn, so a resume works even when every agent is rebuilt from storage on the next request. With <Syntax py="preserve_context=True" ts="preserveContext: true" /> the sub-agent owns its own state: give it its own session manager if the resume has to survive a restart, otherwise the interrupt can only be resumed by the same process that raised it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe put this under an advanced section? This feels like an implementation detail, and I dont think needs to be up front to customers.

carried = {
key: value for key, value in agent._interrupt_state.context.items() if key not in _LOOP_OWNED_CONTEXT_KEYS
}
agent._interrupt_state.context = {**carried, "tool_use_message": message, "tool_results": tool_results}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The **carried is really the actual bug fix in this pull request right? We are propagating context from a tool that was previously not serialized and lost, right?

Do we need to worry about carried being serializable?

Comment on lines +380 to +387
logger.error(
"tool_name=<%s>, agent_name=<%s>, tool_use_id=<%s> | cannot apply the interrupt "
"response yet, raising the interrupt again so it survives to be answered once "
"more",
self._tool_name,
self._agent_name,
tool_use_id,
)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this error happen when a agent-as-tool has multiple interrupts, and only some of them have been responded to? If so, could we mention that in the error message?

Suggested change
logger.error(
"tool_name=<%s>, agent_name=<%s>, tool_use_id=<%s> | cannot apply the interrupt "
"response yet, raising the interrupt again so it survives to be answered once "
"more",
self._tool_name,
self._agent_name,
tool_use_id,
)
logger.error(
"tool_name=<%s>, agent_name=<%s>, tool_use_id=<%s> | cannot apply the interrupt "
"response yet, there are still unanswered interrupts (maybe print interrupt ids here as well), raising the interrupt again so it survives to be answered once "
"more",
self._tool_name,
self._agent_name,
tool_use_id,
)

Comment on lines +392 to +393
"tool_name=<%s>, agent_name=<%s>, tool_use_id=<%s> | cannot resume: the sub-agent's "
"interrupted turn is not available, so the interrupt response cannot be applied",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this happens if an orchestrator with a session manager uses a sub-agent that has preserve_context=True, but no session manager. If the sub-agent interrupts, and then the application stops and restores through session, then the application breaks here.

Is there some way we can prevent this early with errors? Or maybe pass session manager to the child agent? I guess returning a toolResultEvent does help avoid this, but it feels like its just masking the underlying issue that would keep coming up.

if self._preserve_context:
if getattr(self._agent, "_session_manager", None) is None:
logger.warning(
"tool_name=<%s>, agent_name=<%s>, tool_use_id=<%s> | interrupted sub-agent uses "

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow up from related comment, do we just want to throw in this case, or maybe inherit the parent agents session manager?


try:
self._agent.load_snapshot(Snapshot.from_dict(turn))
except Exception as error:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we choose a more specific error here?

Comment on lines +363 to +364
if parent_call is not None and parent_call.awaiting_resume:
if not self._reinstate_turn(parent_call) and not self._agent._interrupt_state.activated:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im having a bit of trouble following the logic here:

In the documentation, you mention that interrupts are stored in different locations if preserve_context is set to true or false right? Where in this code block are we checking the interrupt context if preserve_context=False or the agents sessions if preserve_context=True?

Is there an opportunity to restructure the logic here to make it a bit easier to follow what is going in the different code paths?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-hil Human in the loop and suspend/resume area-multiagent Multi-agent related bug Something isn't working complexity/high A touched function exceeds cognitive complexity 25; may be worth splitting python Pull requests that update python code size/m

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Nested agent-as-tool interrupts don't resume across rehydration (stateless / distributed execution)

5 participants