Skip to content

[Bug]: reasoning parsers silently drop buffered partial delimiters at end of stream (no EOF flush hook) #1998

Description

@ighutake-debug

Bug Description

All reasoning parsers can silently lose trailing text when a streamed response ends with a proper prefix of a delimiter token. The streaming pipeline only invokes parse_reasoning_streaming_incremental for decoded chunks and provides no end-of-stream flush, so any text a parser is holding in its internal buffer at stream end is never emitted.

Two concrete shapes:

  1. BaseReasoningParser (used by deepseek_r1, qwen3, kimi, kimi_k25, kimi_thinking, glm45, step3, …): if the stream ends while the buffer is a partial token (e.g. </th), is_partial_token returns true forever and the fragment is dropped (crates/reasoning_parser/src/parsers/base.rs:90-93).
  2. kimi_k2 (feat(reasoning): add unified kimi_k2 reasoning parser #1992): the trailing-partial hold-back that keeps split </think> / <|tool_calls_section_begin|> markers intact between chunks also holds a genuine stream-ending fragment (e.g. reasoning whose final character is <, or output truncated at </thi).

This also makes streaming disagree with one-shot behavior: detect_and_parse_reasoning on the same truncated input emits everything as reasoning.

Steps to Reproduce

let mut p = KimiK2Parser::new();
p.parse_reasoning_streaming_incremental("some reasoning</thi").unwrap();
// stream ends here (no more chunks) — "</thi" is never emitted

Expected Behavior

Held text is flushed when the stream ends: the pipeline calls a parser finalization hook at end-of-stream, and parsers emit any buffered remainder.

Actual Behavior

Buffered partial-delimiter text is silently lost.

Component

protocols (API types) / reasoning parser

Proposed Fix

  • Add a finalization method to the ReasoningParser trait, e.g. fn flush(&mut self) -> Result<ParserResult, ParseError> with a default implementation returning the buffered remainder (or ParserResult::default() for parsers that hold nothing).
  • Call it from the gRPC streaming response paths after the last decoded chunk.
  • Update BaseReasoningParser and kimi_k2 to emit their held buffers from flush().

Surfaced during review of #1992 (codex P2 comment); kept out of that PR's scope per one-concern-per-PR.

Pre-submission Checklist

  • I have searched existing issues and discussions
  • I can reproduce this issue consistently
  • I am using the latest version of SMG

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions