gateway: thread --timeout flag into DispatcherManager (fixes #894)#895
Closed
Harishkrishna17 wants to merge 1 commit into
Closed
Conversation
…ph-ai#894) The api-gateway accepts a --timeout flag (default 600s) but DispatcherManager never received it and instantiated every per-service dispatcher with a hardcoded timeout=120. The flag was therefore silently ignored for all per-service paths (graph-rag, document-rag, librarian, etc.); requests over 120s returned {"error":{"type":"gateway-error","message":"Timeout"}} regardless of the configured value. This adds a timeout parameter to DispatcherManager.__init__ (default 120 to preserve existing behaviour for callers that don't pass it), stores it on self.timeout, uses it in both dispatcher construction sites, and threads self.timeout through from gateway/service.py where DispatcherManager is instantiated. The default of 120 is preserved so this is a non-breaking change. Callers that already pass timeout (none upstream today) continue to work.
Contributor License AgreementThank you for your contribution! Before we can accept it, the following contributor(s) must sign our CLA: Please read the appropriate agreement:
Once you have read the appropriate agreement, post the following as a comment on this PR (copy and paste exactly): The bot will record your signature and update this PR automatically. |
Contributor
|
Possible overlap: #931 |
Contributor
|
Stale, and possibly fixed elsewhere |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #894.
Problem
The
api-gatewayaccepts a--timeoutflag (default600,gateway/service.py:36)documented as "API request timeout in seconds", but the value
never reaches
DispatcherManager, which constructs every per-servicedispatcher with a hardcoded
timeout = 120. So--timeout 600(orany other value) had no effect on graph-rag, document-rag, librarian,
text-completion, or any other request/response path. Long-running
requests deterministically returned
{"error":{"type":"gateway-error","message":"Timeout"}}at exactly2:00 wallclock, regardless of the gateway flag.
The WebSocket endpoint manager already received
self.timeoutcorrectly; only the dispatcher manager was missing the wiring.
Change
timeout=120parameter toDispatcherManager.__init__(default preserves prior behaviour for any external caller).
self.timeout.self.timeoutin both dispatcher construction sites(
global_dispatchersat line ~301 andrequest_response_dispatchersat line ~458) instead of the literal
120.timeout=self.timeoutfromApiingateway/service.py:116-122so the existing--timeoutCLI flagfinally reaches the dispatchers.
Compatibility
Non-breaking: default of
120is preserved on the new parameter, soany caller that doesn't pass
timeoutkeeps the previous behaviour.Test plan
python -m py_compile).--timeout 600:curl … /api/v1/flow/default/service/graph-rag …returnsgateway-error: Timeoutat 120s.running container before forking), the same query is allowed
to run to completion (~25s server-side) and returns the
grounded answer.
tests still pass; I haven't run the suite locally because
this is targeted at
release/v2.4which I don't have acomplete test env for.
Related
adjacent. The gateway side is now done; bootstrapper / librarian /
config-client items in Hardcoded timeout values should be configurable #874 remain.