feat(serve): add SSE streaming support for /v1/chat/completions

[tangxinyao]

Replace the hardcoded 400 rejection of stream=true with a StreamingResponse that emits OpenAI-compatible SSE chunks.

The engine now supports true token-level streaming through a unified generate_rollout_stream_async interface. On the CUDA path, worker processes push incremental StreamTokenStep data through multiprocessing.Pipe from within the decode loop, coordinated by TPCluster.stream_call_async. On the MLX path, the ContinuousBatchScheduler daemon thread writes tokens to a queue.Queue, consumed by the serve layer via run_in_executor. Both paths deliver tokens as they are produced, providing low time-to-first-token and real-time SSE output. Multi-choice (n>1) and tool-call requests fall back to the synchronous generation path with _build_sse_chunks for SSE encoding.

Changes:

• Engine data layer (engine/data/batch.py): add StreamTokenStep — incremental token unit with prompt_idx, token_id, finish_reason
• IPC protocol (engine/protocol.py): add Op.INFER_ROLLOUT_STREAM, StreamTokenPayload, and TPCluster.stream_call_async() — creates mp.Pipe, broadcasts send-end to workers, reads tokens from recv-end in a thread pool as an async generator
• Worker (engine/worker.py): add ArenoWorker.handle_stream_rollout() — extracts pipe send-end, sends ack immediately so coordinator can start reading, drives decode with a stream_callback closure that writes each StreamTokenStep to the pipe
• Decode loop (engine/inference.py): add StreamTokenCallback type and stream_callback parameter to infer_rollout / _generate_rollout_tokens_no_sync — invoked after prefill and after each decode step (TP rank-0 only)
• Engine API (engine/api.py): add ArenoEngine.generate_rollout_stream_async() → AsyncGenerator[StreamTokenStep]
• Serve runtime adapters (cli/serve.py):
  • _CudaServeRuntime.generate_rollout_stream_async: converts areno.api.SamplingParams → engine SamplingParams, delegates to ArenoEngine.generate_rollout_stream_async, adapts StreamTokenStep → _ServeStreamStep
  • _MlxServeRuntime.generate_rollout_stream_async: consumes queue.Queue from ContinuousBatchScheduler.submit_stream() via run_in_executor, wraps as _ServeStreamStep
  • Both adapters share the same signature — serve layer calls state.engine.generate_rollout_stream_async() without isinstance dispatch
• SSE encoding (cli/serve.py):
  • Add ChatCompletionStreamDelta / ChatCompletionStreamChoice Pydantic models
  • Add _stream_chat_completions() — unified SSE entry for both CUDA and MLX, performs incremental prefix decoding and emits SSE chunks token-by-token
  • Add _build_sse_chunks() — fallback SSE builder for multi-choice (n>1) and tool-call requests, using the synchronous generation path
• HTTP handler: branch /v1/chat/completions on request.stream, return StreamingResponse for streaming requests
• Add CPU tests covering text streaming, tool-call deltas, multi-choice, empty content, and SSE formatting edge cases

What does this PR do?

This PR adds SSE streaming support to areno serve's /v1/chat/completions endpoint.

Previously, requests with stream=true were rejected with a 400 error. This prevented AReno from being used as a backend by Agent frameworks that rely on the OpenAI streaming API for tool-use and multi-turn agent execution.

The implementation adds a true token-level streaming pipeline across the full stack:

• Engine decode loop: a stream_callback is invoked after each decode step (both after prefill and after every subsequent token), pushing incremental StreamTokenStep data before the full rollout completes.
• IPC transport (CUDA): multiprocessing.Pipe carries tokens from the worker process to the coordinator, with the worker sending an immediate ack so the coordinator can start reading before decode finishes.
• In-process transport (MLX): queue.Queue bridges the daemon decode thread and the serve layer's async event loop via run_in_executor.
• Serve layer: a unified _stream_chat_completions() coroutine consumes the async generator from either backend, performs incremental prefix decoding, and emits OpenAI-compatible SSE chunks in real time. Multi-choice (n>1) and tool-call requests fall back to the synchronous generation path with _build_sse_chunks for SSE encoding.

This provides both protocol compatibility for Agent frameworks (OpenAI SDK, LangChain, AutoGen, CrewAI) and low time-to-first-token for interactive serving.

Related issue

#503

Type of change

• 🐛 Bug fix
• ✨ New feature
• 💥 Breaking change (public API / CLI behavior changes in a non-backward-compatible way)
• 📝 Documentation update
• ♻️ Refactoring
• ⚡ Performance improvement
• ✅ Test coverage improvement

How was it tested?

Added CPU tests covering:

• Initial assistant role chunk
• Text delta chunking and reconstruction
• finish_reason only appearing in the final chunk
• usage included in the final chunk
• Tool-call delta structure
• Empty content
• Multiple choices (n > 1)
• SSE data: ...\n\n formatting
• SSE stream termination
• Streaming endpoint behavior

Test command:

pytest tests/test_serve_streaming_cpu.py -v

The tests run on CPU and do not require GPU-specific functionality.

Checklist

• The PR title summarizes the contribution.
• Linked the related issue in the description (if any).
• Existing tests pass (pytest tests/ -k cpu).
• New behavior is covered by tests.
• Described the test commands run and any hardware limitations.
• Public API / CLI changes are additive and backward-compatible (see CONTRIBUTING.md).

Breaking change details

[xsuler]

@tangxinyao Please resolve merge conflicts.