Skip to content

feature: update the request_id handling logic to align with vLLM #472

@KevinCheung2259

Description

@KevinCheung2259

Describe the feature

Align with vllm, retrieve X-Request-Id from request headers as request_id or generating a new uuid as request_id only when X-Request-Id is not present in the request headers.
In vllm/vllm/entrypoints/openai/serving_completion.py

    @staticmethod
    def _base_request_id(raw_request: Optional[Request],
                         default: Optional[str] = None) -> Optional[str]:
        """Pulls the request id to use from a header, if provided"""
        default = default or random_uuid()
        if raw_request is None:
            return default

        return raw_request.headers.get("X-Request-Id", default)

Why do you need this feature?

  1. Improve request tracing in serving systems: Allow upstream services to pass request_id to the router, maintaining request_id consistency throughout the service chain
  2. Simplify troubleshooting: External systems can provide their own request_id, making cross-system tracing and debugging easier.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions