-
Notifications
You must be signed in to change notification settings - Fork 468
fix(langchain): do not read from streamed chunks when there are none for chat model streams #14985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 237 ± 2 ms. The average import time from base is: 241 ± 4 ms. The import time difference between this PR and base is: -4.3 ± 0.1 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate sabrenner/langchain-stream-fix (bbb5b1a) with baseline main (11fdb8b) 🟡 Near SLO Breach (3 suites)🟡 djangosimple - 30/30✅ appsecTime: ✅ 20.552ms (SLO: <22.300ms -7.8%) vs baseline: +0.2% Memory: ✅ 65.247MB (SLO: <67.000MB -2.6%) vs baseline: +4.8% ✅ exception-replay-enabledTime: ✅ 1.350ms (SLO: <1.450ms -6.9%) vs baseline: +0.4% Memory: ✅ 64.691MB (SLO: <67.000MB -3.4%) vs baseline: +4.9% ✅ iastTime: ✅ 20.555ms (SLO: <22.250ms -7.6%) vs baseline: ~same Memory: ✅ 65.220MB (SLO: <67.000MB -2.7%) vs baseline: +4.8% ✅ profilerTime: ✅ 15.499ms (SLO: <16.550ms -6.4%) vs baseline: +0.1% Memory: ✅ 54.146MB (SLO: <54.500MB 🟡 -0.6%) vs baseline: +4.9% ✅ resource-renamingTime: ✅ 20.592ms (SLO: <21.750ms -5.3%) vs baseline: +0.1% Memory: ✅ 65.234MB (SLO: <67.000MB -2.6%) vs baseline: +4.7% ✅ span-code-originTime: ✅ 25.425ms (SLO: <28.200ms -9.8%) vs baseline: ~same Memory: ✅ 67.589MB (SLO: <69.500MB -2.7%) vs baseline: +4.9% ✅ tracerTime: ✅ 20.547ms (SLO: <21.750ms -5.5%) vs baseline: +0.7% Memory: ✅ 65.313MB (SLO: <67.000MB -2.5%) vs baseline: +4.9% ✅ tracer-and-profilerTime: ✅ 22.639ms (SLO: <23.500ms -3.7%) vs baseline: ~same Memory: ✅ 66.744MB (SLO: <67.500MB 🟡 -1.1%) vs baseline: +4.9% ✅ tracer-dont-create-db-spansTime: ✅ 19.386ms (SLO: <21.500ms -9.8%) vs baseline: +0.3% Memory: ✅ 65.254MB (SLO: <66.000MB 🟡 -1.1%) vs baseline: +4.9% ✅ tracer-minimalTime: ✅ 16.620ms (SLO: <17.500ms -5.0%) vs baseline: -0.1% Memory: ✅ 65.274MB (SLO: <66.000MB 🟡 -1.1%) vs baseline: +4.9% ✅ tracer-nativeTime: ✅ 20.499ms (SLO: <21.750ms -5.8%) vs baseline: -0.4% Memory: ✅ 71.467MB (SLO: <72.500MB 🟡 -1.4%) vs baseline: +4.8% ✅ tracer-no-cachesTime: ✅ 18.599ms (SLO: <19.650ms -5.3%) vs baseline: +0.4% Memory: ✅ 65.254MB (SLO: <67.000MB -2.6%) vs baseline: +4.8% ✅ tracer-no-databasesTime: ✅ 18.795ms (SLO: <20.100ms -6.5%) vs baseline: +0.1% Memory: ✅ 65.268MB (SLO: <67.000MB -2.6%) vs baseline: +4.8% ✅ tracer-no-middlewareTime: ✅ 20.227ms (SLO: <21.500ms -5.9%) vs baseline: +0.1% Memory: ✅ 65.274MB (SLO: <67.000MB -2.6%) vs baseline: +4.9% ✅ tracer-no-templatesTime: ✅ 20.386ms (SLO: <22.000ms -7.3%) vs baseline: -0.3% Memory: ✅ 65.234MB (SLO: <67.000MB -2.6%) vs baseline: +4.7% 🟡 errortrackingdjangosimple - 6/6✅ errortracking-enabled-allTime: ✅ 18.066ms (SLO: <19.850ms -9.0%) vs baseline: -0.4% Memory: ✅ 65.223MB (SLO: <66.500MB 🟡 -1.9%) vs baseline: +4.8% ✅ errortracking-enabled-userTime: ✅ 18.153ms (SLO: <19.400ms -6.4%) vs baseline: +0.4% Memory: ✅ 65.226MB (SLO: <66.500MB 🟡 -1.9%) vs baseline: +4.8% ✅ tracer-enabledTime: ✅ 18.061ms (SLO: <19.450ms -7.1%) vs baseline: -0.4% Memory: ✅ 65.199MB (SLO: <66.500MB 🟡 -2.0%) vs baseline: +4.9% 🟡 flasksimple - 18/18✅ appsec-getTime: ✅ 4.601ms (SLO: <4.750ms -3.1%) vs baseline: +0.2% Memory: ✅ 62.030MB (SLO: <65.000MB -4.6%) vs baseline: +5.0% ✅ appsec-postTime: ✅ 6.613ms (SLO: <6.750ms -2.0%) vs baseline: -0.5% Memory: ✅ 61.867MB (SLO: <65.000MB -4.8%) vs baseline: +4.7% ✅ appsec-telemetryTime: ✅ 4.598ms (SLO: <4.750ms -3.2%) vs baseline: +0.2% Memory: ✅ 61.951MB (SLO: <65.000MB -4.7%) vs baseline: +4.8% ✅ debuggerTime: ✅ 1.855ms (SLO: <2.000ms -7.3%) vs baseline: -0.2% Memory: ✅ 45.377MB (SLO: <47.000MB -3.5%) vs baseline: +4.7% ✅ iast-getTime: ✅ 1.857ms (SLO: <2.000ms -7.2%) vs baseline: -0.6% Memory: ✅ 42.369MB (SLO: <49.000MB 📉 -13.5%) vs baseline: +4.8% ✅ profilerTime: ✅ 1.911ms (SLO: <2.100ms -9.0%) vs baseline: -0.4% Memory: ✅ 46.498MB (SLO: <47.000MB 🟡 -1.1%) vs baseline: +4.6% ✅ resource-renamingTime: ✅ 3.376ms (SLO: <3.650ms -7.5%) vs baseline: +0.1% Memory: ✅ 52.219MB (SLO: <53.500MB -2.4%) vs baseline: +4.8% ✅ tracerTime: ✅ 3.358ms (SLO: <3.650ms -8.0%) vs baseline: +0.2% Memory: ✅ 52.298MB (SLO: <53.500MB -2.2%) vs baseline: +4.9% ✅ tracer-nativeTime: ✅ 3.358ms (SLO: <3.650ms -8.0%) vs baseline: ~same Memory: ✅ 58.119MB (SLO: <60.000MB -3.1%) vs baseline: +4.8%
|
…d-trace-py into sabrenner/langchain-stream-fix
…for chat model streams (#14985) ## Description Fixes an issue where an `IndexError` would be raised trying to access a chunk that didn't exist when a streamed chat model operation did not produce any chunks - usually because of a timeout error. ## Testing Verified with the reproduction in the linked issue ## Risks None ## Additional Notes Closes #14688 (cherry picked from commit 99b19b7)
…for chat model streams (#14985) ## Description Fixes an issue where an `IndexError` would be raised trying to access a chunk that didn't exist when a streamed chat model operation did not produce any chunks - usually because of a timeout error. ## Testing Verified with the reproduction in the linked issue ## Risks None ## Additional Notes Closes #14688 (cherry picked from commit 99b19b7)
…for chat model streams [backport 3.17] (#15058) Backport 99b19b7 from #14985 to 3.17. ## Description Fixes an issue where an `IndexError` would be raised trying to access a chunk that didn't exist when a streamed chat model operation did not produce any chunks - usually because of a timeout error. ## Testing Verified with the reproduction in the linked issue ## Risks None ## Additional Notes Closes #14688 Co-authored-by: Sam Brenner <[email protected]>
…for chat model streams (#14985) ## Description Fixes an issue where an `IndexError` would be raised trying to access a chunk that didn't exist when a streamed chat model operation did not produce any chunks - usually because of a timeout error. ## Testing Verified with the reproduction in the linked issue ## Risks None ## Additional Notes Closes #14688
Description
Fixes an issue where an
IndexErrorwould be raised trying to access a chunk that didn't exist when a streamed chat model operation did not produce any chunks - usually because of a timeout error.Testing
Verified with the reproduction in the linked issue
Risks
None
Additional Notes
Closes #14688