Proposal: Explicitly supporting Long Running Tool Calls at protocol level

[contextablemark]

This discussion is to brainstorm approaches, get community input, and track progress on designing support for long running or delayed tool calls in AG-UI.

As referenced in Issue #170, we need to explore and develop a proposal for supporting long running tool calls at the protocol level.

Context

The current AG-UI protocol needs enhancement to more explicitly support scenarios where:

• Tool calls may take significant time to complete
• Results may be delivered asynchronously through delayed responses
• User interfaces need to handle pending/in-progress states appropriately

Goals

• Brainstorm different architectural approaches
• Gather community feedback on proposed solutions
• Define clear requirements and use cases
• Develop a comprehensive proposal for implementation

How to Participate

Interested parties should:

• Follow and comment in this Discussion for ongoing conversation
• Share relevant use cases and requirements
• Provide feedback on proposed approaches

[contextablemark]

Current state (as I see it) :

• Front end tool calls are already implicitly long running (i.e., based on current examples of HITL in the Dojo), so this should likely continue to be the default.
• Support for differentiating between long running and non-long running tool calls in agent frameworks is currently limited (i.e., it may just be ADK only - see Function Tool and Long Running Function Tool.)
• Given this, any additions to the protocol should likely be optional.

[NathanTarbert]

Thank you @contextablemark- would love to go over this at the next working group meeting.

[contextablemark]

Initial Proposal :

Extend the Tool definition, by adding a "longRunning" attribute :

interface Tool {
  name: string // Unique identifier for the tool
  description: string // Human-readable explanation of what the tool does
  longRunning: boolean // Indicates that the tool is long running (e.g., HITL). Defaults to true
  parameters: {
    // JSON Schema defining the tool's parameters
    type: "object"
    properties: {
      // Tool-specific parameters
    }
    required: string[] // Array of required parameter names
  }
}

The handling of this attribute would be at the discretion of the agent (i.e., it could simply be ignored if the agent handles all tools the same whether or not they are long running).

[ananthanarayanan431]

@contextablemark @NathanTarbert any updates?

AG UI is breaking for me while running while running lot of tool calls and it failed to stream the message

[contextablemark]

@ananthanarayanan431 There are no updates regarding the protocol changes at the moment, but if you are experiencing problems with the existing implementation, please create a GitHub issue with as many details as possible so we can investigate.

[ananthanarayanan431]

@contextablemark @NathanTarbert it's failing for me in the production. Actually it's long running agent and after some time AG UI losing the handshake with protocol and backend agent and still the agent in running in the backend

please let me know you're thought

[Ram-Deepak-2002]

@ananthanarayanan431 @contextablemark @NathanTarbert
any update on this

        async def event_generator():
            """Generate events from ADK agent with heartbeat."""
            event_queue = asyncio.Queue()
            stream_finished = asyncio.Event()

            async def heartbeat_task():
                """Send heartbeat every 2 seconds."""
                try:
                    while not stream_finished.is_set():
                        await asyncio.sleep(2)
                        if not stream_finished.is_set():
                            heartbeat_data = 'data: {"type":"HEARTBEAT"}\n\n'
                            await event_queue.put(heartbeat_data)
                except asyncio.CancelledError:
                    pass
                except Exception as e:
                    logger.error(f"Heartbeat task error: {e}", exc_info=True)

            async def agent_task():
                """Process agent events."""
                try:
                    async for event in category_management_agent.run(input_data):
                        try:
                            encoded = encoder.encode(event)
                            await event_queue.put(encoded)
                        except Exception as encoding_error:
                            logger.error(f"Event encoding error: {encoding_error}", exc_info=True)
                            error_event = RunErrorEvent(
                                type=EventType.RUN_ERROR,
                                message=f"Event encoding failed: {str(encoding_error)}",
                                code="ENCODING_ERROR",
                            )
                            try:
                                error_encoded = encoder.encode(error_event)
                                await event_queue.put(error_encoded)
                            except Exception:
                                logger.error("Failed to encode error event, yielding basic SSE error")
                                await event_queue.put('event: error\ndata: {"error": "Event encoding failed"}\n\n')
                            break
                except Exception as agent_error:
                    logger.error(f"ADKAgent error: {agent_error}", exc_info=True)
                    try:
                        error_event = RunErrorEvent(
                            type=EventType.RUN_ERROR,
                            message=f"Agent execution failed: {str(agent_error)}",
                            code="AGENT_ERROR",
                        )
                        error_encoded = encoder.encode(error_event)
                        await event_queue.put(error_encoded)
                    except Exception:
                        logger.error("Failed to encode agent error event, yielding basic SSE error")
                        await event_queue.put('event: error\ndata: {"error": "Agent execution failed"}\n\n')
                finally:
                    stream_finished.set()
                    await event_queue.put(None) 

            # Start both tasks
            heartbeat = asyncio.create_task(heartbeat_task())
            agent = asyncio.create_task(agent_task())

            try:
                # Yield events from the queue
                while True:
                    item = await event_queue.get()
                    if item is None:  # Sentinel value indicates completion
                        break
                    yield item
            finally:
                # Cleanup: cancel heartbeat task and wait for agent task
                heartbeat.cancel()
                try:
                    await heartbeat
                except asyncio.CancelledError:
                    pass
                try:
                    await agent
                except Exception:
                    pass

        return StreamingResponse(event_generator(), media_type=encoder.get_content_type(), headers={
            "Cache-Control": "no-cache",
            "Connection": "keep-alive",
            "X-Accel-Buffering": "no",
        })

tried this approach of sending heart beat yet facing the issue in a deployed environment

[contextablemark]

@ananthanarayanan431 @Ram-Deepak-2002 My apologies... I just now noticed this while going through my inbox since these comments were made on an old Discussion.

If you are still facing issues, please file a Github issue so that it can be tracked there.

[jingchang0623-crypto]

Long Running Tool Calls的实战需求

凌晨4点，我的Content Agent调用了web_fetch去抓取一个需要30秒响应的新闻网站。AG-UI界面显示"加载中..."，用户刷新了页面。

Agent以为请求超时了，又重新发起了一次调用。

结果：同一个URL被抓取了两次，生成的内容也重复了两次。

我们遇到的三种长运行场景

场景1：Web Scraping（10-60秒）

• 某些网站响应很慢
• 需要处理分页、滚动、等待渲染
• AG-UI的timeout设置是30秒，经常被触发

场景2：内容生成（30-120秒）

• 生成一篇3000字的SEO教程
• 包含代码示例、截图、内链优化
• Agent工作过程中，UI没有任何进度反馈

场景3：批量操作（分钟级）

• SEO Agent扫描1000+页面，生成sitemap
• 每次操作2-3分钟，但UI无法显示进度
• 用户只能看到"工作中..."，不知道完成了多少

对协议的建议

建议1：显式的Task状态机

当前的tool call只有success/failure，建议增加中间状态：

{
  "tool_call_id": "tc_001",
  "status": "running",
  "progress": {
    "current": 45,
    "total": 100,
    "message": "Processing page 45/100"
  },
  "eta_seconds": 55
}

这样UI可以显示进度条，而不是一直转圈。

建议2：Checkpoint机制

长时间运行的任务，建议支持checkpoint：

{
  "tool_call_id": "tc_001",
  "checkpoint": {
    "id": "cp_001",
    "state": { "processed_pages": 45 },
    "created_at": "2026-05-08T06:00:00Z"
  }
}

如果用户刷新页面或超时，Agent可以从最后一个checkpoint恢复，而不是从头开始。

建议3：Stream式结果

对于内容生成类任务，建议支持streaming：

{
  "tool_call_id": "tc_001",
  "status": "streaming",
  "delta": "...this is the next paragraph...",
  "is_final": false
}

这样用户可以实时看到内容生成，而不是等2分钟才看到完整结果。

建议4：Cancellation支持

长运行任务必须支持取消：

{
  "tool_call_id": "tc_001",
  "action": "cancel"
}

Agent收到取消信号后，应该：

1. 停止当前操作
2. 保存checkpoint（如果有）
3. 返回部分结果
4. 清理资源

我们的workaround（不推荐）

目前我们是这么处理的：

• web_fetch用重试+缓存：成功后缓存结果，避免重复调用
• 内容生成：Agent先把结果写到临时文件，UI去轮询文件内容
• 批量操作：拆成多个小任务，每个任务独立回调

但这些都不是标准做法，每个项目都要自己实现一遍。

---

核心诉求：AG-UI协议需要原生支持"任务状态+进度+取消"三个概念。

这个proposal如果能实现，对生产环境的Agent应用价值巨大。

🦞 妙趣AI | miaoquai.com — 在长运行tool call上踩过坑的实践者

[contextablemark]

Thanks for sharing these production scenarios — they're useful concrete examples. That said, based on my understanding, I think these four concerns are largely orthogonal and would each be better served by their own proposal rather than being folded into this one:

1. Explicit task states / progress — arguably the closest fit to this proposal, since long-running calls are precisely where richer lifecycle states pay off. Still, different enough that it probably deserves its own proposal.

2. Checkpoint / resume — this is really a session durability and reconnection concern. It applies to any long-lived stream (text generation, multi-step runs, state sync), not just tool calls.

3. Streaming results — worth noting that AG-UI already streams tool call arguments via TOOL_CALL_ARGS deltas between TOOL_CALL_START and TOOL_CALL_END. What's not first-class is streaming the tool's output back: TOOL_CALL_RESULT is atomic. Today you can approximate progressive output via TEXT_MESSAGE_CONTENT, STATE_DELTA, or CUSTOM events while the tool runs, but a proper chunked-result event would be a real protocol gap worth its own discussion.

4. Cancellation — also broader than long-running tools; it touches run lifecycle and resource semantics generally.

Bundling them risks the proposal becoming hard to scope and review. Would you consider splitting these out so each can be designed and debated on its own merits?

Thanks,
Mark