feat(serve): add SSE streaming support for /v1/chat/completions - #504

Open

tangxinyao wants to merge 4 commits into

inclusionAI:mainfrom

tangxinyao:fix/serve-stream-support

Commits on Aug 24, 2026

feat(serve): add SSE streaming support for /v1/chat/completions

tangxinyao
and
claude
committed
7e41d28
feat(serve): unify generate_rollout_stream_async interface across CUDA and MLX paths

tangxinyao
and
claude
committed
54929a4

Commits on Aug 25, 2026

feat(serve): stream continuous batches over shared result_queue

tangxinyao
and
claude
committed
9ed73b6

Commits on Aug 27, 2026

Merge branch 'main' into fix/serve-stream-support
tangxinyao
committed
fc68567