Context assembler can emit a 'tool' message with no preceding 'tool_calls' carrier (HTTP 400 from OpenAI/Azure)

[sahulphaniraj-cisco]

Summary

assemble_llm_messages() can emit a messages[] payload that violates the OpenAI tool-call contract: a role: "tool" message with no preceding assistant message carrying tool_calls. Strict providers reject the whole request:

HTTP 400: AzureException BadRequestError - Invalid parameter: messages with role 'tool'
must be a response to a preceeding message with 'tool_calls'.

WorkingMemoryContextSource.collect_context() already protects its own window from splitting a tool-call group (working_memory.py:117-122). Committed history gets no equivalent guard, and the assembled payload is never validated before it goes out.

Reproduction

No credentials, no manifest, no LLM call. Save as repro_orphan_tool_message.py and run with $MAS_LAB_OSS/.venv/bin/python repro_orphan_tool_message.py.

from mas.runtime.boundary.context.assemble import assemble_llm_messages
from mas.runtime.driver.mocks import AutoCtxAssembler


def first_violation(messages):
    """Return (index, preceding_role) for the first tool message lacking a carrier."""
    for i, msg in enumerate(messages):
        if msg.get("role") != "tool":
            continue
        j = i
        while j > 0 and messages[j - 1].get("role") == "tool":
            j -= 1
        prev = messages[j - 1] if j > 0 else None
        if not (prev and prev.get("role") == "assistant" and prev.get("tool_calls")):
            return i, (prev or {}).get("role")
    return None, None


def show(label, messages):
    idx, prev_role = first_violation(messages)
    print(f"\n{label}")
    print(f"  roles: {[m.get('role') for m in messages]}")
    if idx is None:
        print("  OK - every tool message follows an assistant message with tool_calls")
    else:
        print(f"  VIOLATION - messages[{idx}] role='tool' preceded by role='{prev_role}'")


def case_turn_commit_between_call_and_result():
    """A turn commit lands between the tool call and its result."""
    ctx = AutoCtxAssembler(injected_context=["You have tools to look up schedules."])
    ctx.note_user_input("Plan a trip to Portalis.")
    ctx.record_assistant_tool_call(
        call_id="call_1", tool_name="delegate_to_schedule_agent", arguments={"q": "times"}
    )
    # Commits the tool_calls carrier into committed_messages, then clears working
    # memory (mocks.py:103) -- the group is now split across the two stores.
    ctx.note_agent_response("Here are the departure times.")
    ctx.record_tool_result(call_id="call_1", content='{"times": ["05:00", "07:30"]}')
    return assemble_llm_messages(ctx)


def case_orphan_persists_in_committed_history():
    """Once committed, the orphan is replayed on every later call for that agent."""
    ctx = AutoCtxAssembler(injected_context=["You have tools to look up schedules."])
    ctx.record_tool_result(call_id="call_1", content='{"found": true}')
    ctx.note_agent_response("")  # commits working memory verbatim (mocks.py:86-87)
    ctx.note_user_input("And what about attractions?")
    ctx.record_assistant_message("Checking attractions.")
    ctx.note_agent_response("Here are the attractions.")
    return assemble_llm_messages(ctx)


if __name__ == "__main__":
    show("case 1 - turn commit splits the tool-call group",
         case_turn_commit_between_call_and_result())
    show("case 2 - orphan persists in committed history",
         case_orphan_persists_in_committed_history())

Actual output — main @ c475d2c, Python 3.13.4, clean tree

case 1 - turn commit splits the tool-call group
  roles: ['system', 'user', 'assistant', 'assistant', 'user', 'tool']
  VIOLATION - messages[5] role='tool' preceded by role='user'

case 2 - orphan persists in committed history
  roles: ['system', 'tool', 'user', 'assistant', 'assistant', 'user']
  VIOLATION - messages[1] role='tool' preceded by role='system'

Expected

Every role: "tool" message in the assembled payload is preceded by an assistant message whose tool_calls contains the matching tool_call_id — or the incomplete group is dropped rather than emitted.

Every call in case 1 is a public runtime API used in a legitimate order, so no misuse is required to reach this state.

Where it comes from

Location	Role in the bug
driver/mocks.py:86-87,103	note_agent_response commits raw working memory, then clears it — nothing keeps a tool-call group intact across the clear
context/assemble.py:72-91	committed history is spliced into messages[] with no pairing validation before the request is sent
context/working_memory.py:117-122	already has the correct guard for its own window — the asymmetry is what makes this look like an oversight rather than a design choice

Note that AutoCtxAssembler is on the production path despite living in mocks.py: it is instantiated by ctl/session/bootstrap.py:70, used as the driver's default ctx (driver/driver.py:97), and by driver/instance.py:60.

Impact

Observed in a real 4-agent run (trip-planner, delegation + tool loop, azure/gpt-4o). Reconstructing all 54 context assemblies from events.jsonl found 2 with the shape [system, tool, assistant, user, ...] — matching case 2 above — and the run log carries the corresponding 400.

Two aggravating factors:

1. The orphan persists. Once it is in committed_messages, StackConversation.max_messages defaults to None, so history is never trimmed and the invalid payload replays on every subsequent LLM call for that agent. In our run the fallback retry failed identically rather than recovering.
2. Lenient providers hide it. The runtime emits an invalid payload regardless of provider; Azure/OpenAI simply validate it. A lenient provider silently accepts a conversation containing a dangling tool result, which is worse for trace fidelity — the agent loses one tool result's worth of grounding with no error surfaced.

In our case the failure was non-fatal (it hit a delegated sub-agent, the orchestrator absorbed it and the run completed), so this can degrade agent reasoning without failing the run.

Additional notes

• Confirmed live end-to-end. Driving the same contexts through LiveLlmEngine.invoke(LLM_CALL) against a real endpoint reproduces the 400, while a control case — identical system prompt, user text, tool call, tool result, model and key, differing only in whether a commit split the group — returns 200. Happy to attach that script; it is omitted here to keep the repro credential-free.
• Two faces of one invariant. Case 1 also leaves a carrier with no response, and providers report whichever half they notice first (...did not have response messages: call_1). A fix should preserve pairing rather than only suppress orphaned tool messages.
• Not proven: the exact production interleaving that split the group. The most likely candidate is a working-memory clear() landing between record_assistant_tool_call and record_tool_result around a delegation boundary, but the trace we have emits only state_update_start/end, not context_mutation, so the timeline needed to confirm it is not there. The synthetic repro above demonstrates the missing invariant deterministically either way.

Possibly related

The max_messages / max_tokens values in our agent manifests sat under a plugin's config: block rather than spec.context_manager, and context_manager_spec() reads only the latter — so those caps were silently inert. Not the cause of this bug, but a related footgun if context caps are expected to apply wherever they are declared.