-
Notifications
You must be signed in to change notification settings - Fork 249
Closed
Labels
feature requestNew feature or requestNew feature or request
Description
Describe the feature
Align with vllm, retrieve X-Request-Id from request headers as request_id or generating a new uuid as request_id only when X-Request-Id is not present in the request headers.
In vllm/vllm/entrypoints/openai/serving_completion.py
@staticmethod
def _base_request_id(raw_request: Optional[Request],
default: Optional[str] = None) -> Optional[str]:
"""Pulls the request id to use from a header, if provided"""
default = default or random_uuid()
if raw_request is None:
return default
return raw_request.headers.get("X-Request-Id", default)
Why do you need this feature?
- Improve request tracing in serving systems: Allow upstream services to pass request_id to the router, maintaining request_id consistency throughout the service chain
- Simplify troubleshooting: External systems can provide their own request_id, making cross-system tracing and debugging easier.
Additional context
No response
Metadata
Metadata
Assignees
Labels
feature requestNew feature or requestNew feature or request