Skip to content

Conversation

@ngxson
Copy link
Owner

@ngxson ngxson commented Nov 14, 2025

In this PR, the /v1/chat/completion handler is removed and replaced by a proxy function that will proxy the request to /chat/completion endpoint.

This acts as a demo on how the streamed proxy function can work.

Currently, it works with web UI. But it crashes when the request is terminated unexpectedly (i.e. user clicks on Stop button on web)

Tested so far these cases:

  • normal case (both ends of the proxy open and terminate gracefully)
  • the destination server does not exist --> return 500 code to client
  • the client terminates unexpectedly (for example, user clicks on Stop on webui)
  • the destination server terminates unexpectedly

@coderabbitai
Copy link

coderabbitai bot commented Nov 14, 2025

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch xsn/poc_proxy_2

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants