Skip to content

feat(framework): Add non-streaming Runtime Responses endpoint - #7978

Draft
charlesbvll wants to merge 7 commits into
mainfrom
runtime-responses-endpoint-non-streaming
Draft

feat(framework): Add non-streaming Runtime Responses endpoint#7978
charlesbvll wants to merge 7 commits into
mainfrom
runtime-responses-endpoint-non-streaming

Conversation

@charlesbvll

Copy link
Copy Markdown
Member

Adds a non-streaming Open Responses-compatible endpoint at POST /v1/runtime/responses.

It lets AgentApps use the OpenAI SDK with a Runtime-provided URL and task token while reusing the existing child model-task execution path. The endpoint authenticates the AgentApp, creates a model task, sends the request, waits for its correlated response, and returns it as JSON.

stream=True is explicitly rejected for now. Existing agent.responses.create(...) behavior and UI event streaming remain unchanged.

…point-non-streaming

# Conflicts:
#	framework/py/flwr/supercore/servicer/runtime/runtime_handlers_test.py
#	framework/py/flwr/supercore/task_process/agent/session.py
Copilot AI lite review requested due to automatic review settings August 22, 2026 16:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a non-streaming Open Responses-compatible POST /v1/runtime/responses endpoint for authenticated AgentApps.

Changes:

  • Routes requests through existing model-task execution.
  • Exposes Runtime URL, task token, and CA configuration.
  • Registers the router and adds message conversion and tests.
  • Rejects stream=True.

Outstanding findings:

  • Moderate: Add a total response timeout after a child task reaches RUNNING.
  • Critical: Validate that replies match the outbound request message ID.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Summary
framework/py/flwr/superlink/routers/runtime/responses.py Runtime Responses endpoint and task exchange logic
framework/py/flwr/superlink/routers/runtime/responses_test.py Endpoint and lifecycle tests
framework/py/flwr/superlink/routers/runtime/__init__.py Router export
framework/py/flwr/superlink/main.py Router registration
framework/py/flwr/supercore/task_process/agent/run_agentapp.py Runtime environment and CA setup
framework/py/flwr/supercore/task_process/agent/run_agentapp_test.py Environment configuration tests
framework/py/flwr/supercore/json_message/model_message.py Model request construction
framework/py/flwr/supercore/json_message/model_message_test.py Model message tests
Suppressed comments (2)

framework/py/flwr/supercore/json_message/model_message.py:82

  • This new payload path passes max_output_tokens through the generic integer validator, but that validator uses isinstance(value, int) and therefore accepts JSON booleans. A request such as {"max_output_tokens": true} is forwarded to the model provider instead of rejected as invalid input, typically becoming a 502 provider error. Reject booleans explicitly in the validator (or in ModelRequest validation) before exposing this path.
            max_output_tokens=cast(int | None, payload.get("max_output_tokens")),

framework/py/flwr/superlink/routers/runtime/responses.py:322

  • This cleanup drains only a reply. If the client disconnects or the launch timeout fires before flwr-model pulls the request, the original request remains in task_message with SqlLinkState: its cleanup removes expired rows but not rows whose destination task has been finished, so each abandoned call leaves a database row until the 12-hour TTL. Drain the message addressed to exchange.model_task_id as well (or explicitly delete it) after stopping the child.
def _stop_model_task(state: LinkState, exchange: _Exchange, details: str) -> None:
    """Stop an unfinished model task and drain an already-arrived reply."""
    state.finish_task(exchange.model_task_id, SubStatus.STOPPED, details)
    try:
        _claim_response(state, exchange)

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread framework/py/flwr/superlink/routers/runtime/responses.py Outdated
Comment thread framework/py/flwr/superlink/routers/runtime/responses.py
danielnugraha
danielnugraha previously approved these changes Aug 22, 2026
@github-actions github-actions Bot added the Maintainer Used to determine what PRs (mainly) come from Flower maintainers. label Aug 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Maintainer Used to determine what PRs (mainly) come from Flower maintainers.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants