You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
All reasoning parsers can silently lose trailing text when a streamed response ends with a proper prefix of a delimiter token. The streaming pipeline only invokes parse_reasoning_streaming_incremental for decoded chunks and provides no end-of-stream flush, so any text a parser is holding in its internal buffer at stream end is never emitted.
Two concrete shapes:
BaseReasoningParser (used by deepseek_r1, qwen3, kimi, kimi_k25, kimi_thinking, glm45, step3, …): if the stream ends while the buffer is a partial token (e.g. </th), is_partial_token returns true forever and the fragment is dropped (crates/reasoning_parser/src/parsers/base.rs:90-93).
kimi_k2 (feat(reasoning): add unified kimi_k2 reasoning parser #1992): the trailing-partial hold-back that keeps split </think> / <|tool_calls_section_begin|> markers intact between chunks also holds a genuine stream-ending fragment (e.g. reasoning whose final character is <, or output truncated at </thi).
This also makes streaming disagree with one-shot behavior: detect_and_parse_reasoning on the same truncated input emits everything as reasoning.
Steps to Reproduce
letmut p = KimiK2Parser::new();
p.parse_reasoning_streaming_incremental("some reasoning</thi").unwrap();// stream ends here (no more chunks) — "</thi" is never emitted
Expected Behavior
Held text is flushed when the stream ends: the pipeline calls a parser finalization hook at end-of-stream, and parsers emit any buffered remainder.
Actual Behavior
Buffered partial-delimiter text is silently lost.
Component
protocols (API types) / reasoning parser
Proposed Fix
Add a finalization method to the ReasoningParser trait, e.g. fn flush(&mut self) -> Result<ParserResult, ParseError> with a default implementation returning the buffered remainder (or ParserResult::default() for parsers that hold nothing).
Call it from the gRPC streaming response paths after the last decoded chunk.
Update BaseReasoningParser and kimi_k2 to emit their held buffers from flush().
Surfaced during review of #1992 (codex P2 comment); kept out of that PR's scope per one-concern-per-PR.
Bug Description
All reasoning parsers can silently lose trailing text when a streamed response ends with a proper prefix of a delimiter token. The streaming pipeline only invokes
parse_reasoning_streaming_incrementalfor decoded chunks and provides no end-of-stream flush, so any text a parser is holding in its internal buffer at stream end is never emitted.Two concrete shapes:
BaseReasoningParser(used bydeepseek_r1,qwen3,kimi,kimi_k25,kimi_thinking,glm45,step3, …): if the stream ends while the buffer is a partial token (e.g.</th),is_partial_tokenreturns true forever and the fragment is dropped (crates/reasoning_parser/src/parsers/base.rs:90-93).kimi_k2(feat(reasoning): add unified kimi_k2 reasoning parser #1992): the trailing-partial hold-back that keeps split</think>/<|tool_calls_section_begin|>markers intact between chunks also holds a genuine stream-ending fragment (e.g. reasoning whose final character is<, or output truncated at</thi).This also makes streaming disagree with one-shot behavior:
detect_and_parse_reasoningon the same truncated input emits everything as reasoning.Steps to Reproduce
Expected Behavior
Held text is flushed when the stream ends: the pipeline calls a parser finalization hook at end-of-stream, and parsers emit any buffered remainder.
Actual Behavior
Buffered partial-delimiter text is silently lost.
Component
protocols (API types) / reasoning parser
Proposed Fix
ReasoningParsertrait, e.g.fn flush(&mut self) -> Result<ParserResult, ParseError>with a default implementation returning the buffered remainder (orParserResult::default()for parsers that hold nothing).BaseReasoningParserandkimi_k2to emit their held buffers fromflush().Surfaced during review of #1992 (codex P2 comment); kept out of that PR's scope per one-concern-per-PR.
Pre-submission Checklist