Repository navigation
fix(openai): align first-chunk fields and cover response combinations - #529
Merged
Merged
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
arekay-nv
marked this pull request as draft
October 5, 2026 17:24
arekay-nv
force-pushed
the
arekay/content_type_coverage
branch
from
October 5, 2026 17:53
6967d7c to
b71b593
Compare
arekay-nv
marked this pull request as ready for review
October 5, 2026 21:58
hvagadia
approved these changes
Oct 8, 2026
Event logs written before the aligned first-chunk layout store reasoning responses as a joined str output beside streamed reasoning. The aligned rule treated that str as non-streaming and dropped all content from the TPOT denominator on replay. The current accumulator never produces that shape, so TextModelOutput and the steady-state diagnostics count all of it as tail, restoring each archived run's original TPOT. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
arekay-nv
force-pushed
the
arekay/content_type_coverage
branch
from
October 8, 2026 02:19
eb375b8 to
eefe1b9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mixed OpenAI chat deltas can contain content, reasoning, and tool calls in the same first response chunk. TPOT used to infer which field held the first token from the final fields, which is only right when a response starts with pure reasoning. #526 kept reasoning from mixed deltas (fixing OSL) but kept that inference, so TPOT counted content already delivered with the first token, or dropped tokens that arrived later:
R1 C0 C1C1C0 C1R0 C1C0 C1C0 C1C1+ first tool fragmentAlign entry zero of each streamed
TextModelOutputfield with the first meaningful delta. Fields that start later get an empty first entry; text tails are aggregated and tool-call batches retain their indices for merging. The accumulator keeps a reference to the first delta alongside its existing contributions. Full-output consumers combine all entries, while TPOT skips entry zero independently in every field. No new output class or wire field is added.Existing event logs keep their original TPOT on replay. Earlier revisions stored reasoning responses as a joined
outputstring beside streamedreasoning. The current accumulator never writes that shape, soTextModelOutputand the steady-state diagnostics count all of that string as tail, matching the TPOT those runs reported. This coversscripts/early_stopping_estimate_from_events.pyand the diagnostics CLI. Neither replays tool-call samples, so the older tool-call layout needs no separate rule.Adds 292 response-contract test cases, including:
Also covers replay of the joined-
outputlayout throughTextModelOutputand the offline diagnostics, and updates existing fixtures and offline diagnostics to the aligned-field contract. Follow-up to merged PR #526.Validation:
scripts/early_stopping_estimate_from_events.pygives the same TPOT as the revision that produced it (1111 ns; 2500 ns without the replay rule).🤖 Generated with Claude Code