-
Notifications
You must be signed in to change notification settings - Fork 457
chore(llmobs): missed updates in span kind key changes #14718
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@sabrenner in which pull request was the issue introduced, and did it run the affected tests? If not we should figure out what's missing from the suitespec. |
i think it might've been because #14617 was merged yesterday, which introduced the test and passed at the time, and then #14600 was merged today w/o getting that change from |
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 241 ± 3 ms. The average import time from base is: 243 ± 4 ms. The import time difference between this PR and base is: -2.3 ± 0.2 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate sabrenner/fix-llmobs-span.kind-stuff (3ad35d7) with baseline main (d7e6134) 📈 Performance Regressions (1 suite)📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 4.157µs (SLO: <10.000µs 📉 -58.4%) vs baseline: +0.2% Memory: ✅ 37.650MB (SLO: <39.000MB -3.5%) vs baseline: +5.0% ✅ ospathbasename_noaspectTime: ✅ 1.074µs (SLO: <10.000µs 📉 -89.3%) vs baseline: -0.7% Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +5.2% ✅ ospathjoin_aspectTime: ✅ 6.040µs (SLO: <10.000µs 📉 -39.6%) vs baseline: -0.8% Memory: ✅ 37.611MB (SLO: <39.000MB -3.6%) vs baseline: +4.7% ✅ ospathjoin_noaspectTime: ✅ 2.291µs (SLO: <10.000µs 📉 -77.1%) vs baseline: ~same Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +4.9% ✅ ospathnormcase_aspectTime: ✅ 3.967µs (SLO: <10.000µs 📉 -60.3%) vs baseline: 📈 +14.4% Memory: ✅ 37.631MB (SLO: <39.000MB -3.5%) vs baseline: +4.8% ✅ ospathnormcase_noaspectTime: ✅ 0.563µs (SLO: <10.000µs 📉 -94.4%) vs baseline: -1.8% Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +4.8% ✅ ospathsplit_aspectTime: ✅ 4.719µs (SLO: <10.000µs 📉 -52.8%) vs baseline: -0.6% Memory: ✅ 37.670MB (SLO: <39.000MB -3.4%) vs baseline: +4.9% ✅ ospathsplit_noaspectTime: ✅ 1.586µs (SLO: <10.000µs 📉 -84.1%) vs baseline: -0.8% Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +5.0% ✅ ospathsplitdrive_aspectTime: ✅ 3.682µs (SLO: <10.000µs 📉 -63.2%) vs baseline: +0.4% Memory: ✅ 37.631MB (SLO: <39.000MB -3.5%) vs baseline: +4.6% ✅ ospathsplitdrive_noaspectTime: ✅ 0.693µs (SLO: <10.000µs 📉 -93.1%) vs baseline: -1.6% Memory: ✅ 37.611MB (SLO: <39.000MB -3.6%) vs baseline: +4.7% ✅ ospathsplitext_aspectTime: ✅ 4.498µs (SLO: <10.000µs 📉 -55.0%) vs baseline: -0.4% Memory: ✅ 37.709MB (SLO: <39.000MB -3.3%) vs baseline: +5.0% ✅ ospathsplitext_noaspectTime: ✅ 1.381µs (SLO: <10.000µs 📉 -86.2%) vs baseline: +0.2% Memory: ✅ 37.690MB (SLO: <39.000MB -3.4%) vs baseline: +4.9% 🟡 Near SLO Breach (4 suites)🟡 djangosimple - 30/30✅ appsecTime: ✅ 20.540ms (SLO: <22.300ms -7.9%) vs baseline: +0.2% Memory: ✅ 65.332MB (SLO: <67.000MB -2.5%) vs baseline: +4.6% ✅ exception-replay-enabledTime: ✅ 1.345ms (SLO: <1.450ms -7.3%) vs baseline: -0.4% Memory: ✅ 64.364MB (SLO: <67.000MB -3.9%) vs baseline: +4.7% ✅ iastTime: ✅ 20.436ms (SLO: <22.250ms -8.2%) vs baseline: -0.2% Memory: ✅ 65.357MB (SLO: <67.000MB -2.5%) vs baseline: +4.8% ✅ profilerTime: ✅ 15.302ms (SLO: <16.550ms -7.5%) vs baseline: ~same Memory: ✅ 53.801MB (SLO: <54.500MB 🟡 -1.3%) vs baseline: +5.2% ✅ resource-renamingTime: ✅ 20.540ms (SLO: <21.750ms -5.6%) vs baseline: ~same Memory: ✅ 65.412MB (SLO: <67.000MB -2.4%) vs baseline: +4.7% ✅ span-code-originTime: ✅ 26.079ms (SLO: <28.200ms -7.5%) vs baseline: -0.4% Memory: ✅ 67.517MB (SLO: <69.500MB -2.9%) vs baseline: +4.8% ✅ tracerTime: ✅ 20.500ms (SLO: <21.750ms -5.7%) vs baseline: ~same Memory: ✅ 65.320MB (SLO: <67.000MB -2.5%) vs baseline: +4.6% ✅ tracer-and-profilerTime: ✅ 21.968ms (SLO: <23.500ms -6.5%) vs baseline: -0.2% Memory: ✅ 66.709MB (SLO: <67.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ tracer-dont-create-db-spansTime: ✅ 19.319ms (SLO: <21.500ms 📉 -10.1%) vs baseline: -0.1% Memory: ✅ 65.362MB (SLO: <66.000MB 🟡 -1.0%) vs baseline: +4.7% ✅ tracer-minimalTime: ✅ 16.626ms (SLO: <17.500ms -5.0%) vs baseline: +0.4% Memory: ✅ 65.341MB (SLO: <66.000MB 🟡 -1.0%) vs baseline: +5.2% ✅ tracer-nativeTime: ✅ 20.471ms (SLO: <21.750ms -5.9%) vs baseline: ~same Memory: ✅ 71.269MB (SLO: <72.500MB 🟡 -1.7%) vs baseline: +4.7% ✅ tracer-no-cachesTime: ✅ 18.423ms (SLO: <19.650ms -6.2%) vs baseline: ~same Memory: ✅ 65.343MB (SLO: <67.000MB -2.5%) vs baseline: +4.6% ✅ tracer-no-databasesTime: ✅ 18.790ms (SLO: <20.100ms -6.5%) vs baseline: ~same Memory: ✅ 65.233MB (SLO: <67.000MB -2.6%) vs baseline: +4.9% ✅ tracer-no-middlewareTime: ✅ 20.164ms (SLO: <21.500ms -6.2%) vs baseline: ~same Memory: ✅ 65.365MB (SLO: <67.000MB -2.4%) vs baseline: +4.7% ✅ tracer-no-templatesTime: ✅ 20.374ms (SLO: <22.000ms -7.4%) vs baseline: +0.5% Memory: ✅ 65.382MB (SLO: <67.000MB -2.4%) vs baseline: +4.8% 🟡 errortrackingdjangosimple - 6/6✅ errortracking-enabled-allTime: ✅ 18.069ms (SLO: <19.850ms -9.0%) vs baseline: +0.2% Memory: ✅ 65.313MB (SLO: <66.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ errortracking-enabled-userTime: ✅ 18.119ms (SLO: <19.400ms -6.6%) vs baseline: +0.4% Memory: ✅ 65.352MB (SLO: <66.500MB 🟡 -1.7%) vs baseline: +4.9% ✅ tracer-enabledTime: ✅ 18.053ms (SLO: <19.450ms -7.2%) vs baseline: ~same Memory: ✅ 65.333MB (SLO: <66.500MB 🟡 -1.8%) vs baseline: +4.9% 🟡 flasksimple - 18/18✅ appsec-getTime: ✅ 4.574ms (SLO: <4.750ms -3.7%) vs baseline: +0.4% Memory: ✅ 61.991MB (SLO: <65.000MB -4.6%) vs baseline: +5.0% ✅ appsec-postTime: ✅ 6.570ms (SLO: <6.750ms -2.7%) vs baseline: -0.3% Memory: ✅ 61.971MB (SLO: <65.000MB -4.7%) vs baseline: +4.9% ✅ appsec-telemetryTime: ✅ 4.563ms (SLO: <4.750ms -3.9%) vs baseline: ~same Memory: ✅ 61.971MB (SLO: <65.000MB -4.7%) vs baseline: +5.1% ✅ debuggerTime: ✅ 1.865ms (SLO: <2.000ms -6.8%) vs baseline: +0.7% Memory: ✅ 45.475MB (SLO: <47.000MB -3.2%) vs baseline: +5.0% ✅ iast-getTime: ✅ 1.861ms (SLO: <2.000ms -7.0%) vs baseline: -0.3% Memory: ✅ 42.271MB (SLO: <49.000MB 📉 -13.7%) vs baseline: +4.6% ✅ profilerTime: ✅ 1.916ms (SLO: <2.100ms -8.8%) vs baseline: +0.1% Memory: ✅ 46.498MB (SLO: <47.000MB 🟡 -1.1%) vs baseline: +5.1% ✅ resource-renamingTime: ✅ 3.381ms (SLO: <3.650ms -7.4%) vs baseline: +0.2% Memory: ✅ 52.298MB (SLO: <53.500MB -2.2%) vs baseline: +5.1% ✅ tracerTime: ✅ 3.374ms (SLO: <3.650ms -7.6%) vs baseline: +0.2% Memory: ✅ 52.199MB (SLO: <53.500MB -2.4%) vs baseline: +4.9% ✅ tracer-nativeTime: ✅ 3.373ms (SLO: <3.650ms -7.6%) vs baseline: ~same Memory: ✅ 58.034MB (SLO: <60.000MB -3.3%) vs baseline: +4.5% 🟡 otelspan - 22/22✅ add-eventTime: ✅ 45.314ms (SLO: <47.150ms -3.9%) vs baseline: +0.1% Memory: ✅ 45.272MB (SLO: <47.000MB -3.7%) vs baseline: +4.9% ✅ add-metricsTime: ✅ 323.487ms (SLO: <344.800ms -6.2%) vs baseline: +0.9% Memory: ✅ 553.533MB (SLO: <562.000MB 🟡 -1.5%) vs baseline: +5.1% ✅ add-tagsTime: ✅ 291.051ms (SLO: <314.000ms -7.3%) vs baseline: +0.2% Memory: ✅ 554.836MB (SLO: <563.500MB 🟡 -1.5%) vs baseline: +5.0% ✅ get-contextTime: ✅ 84.890ms (SLO: <92.350ms -8.1%) vs baseline: +2.7% Memory: ✅ 40.249MB (SLO: <46.500MB 📉 -13.4%) vs baseline: +4.7% ✅ is-recordingTime: ✅ 43.116ms (SLO: <44.500ms -3.1%) vs baseline: +0.9% Memory: ✅ 44.613MB (SLO: <47.500MB -6.1%) vs baseline: +4.7% ✅ record-exceptionTime: ✅ 61.684ms (SLO: <67.650ms -8.8%) vs baseline: -0.4% Memory: ✅ 40.616MB (SLO: <47.000MB 📉 -13.6%) vs baseline: +5.1% ✅ set-statusTime: ✅ 48.636ms (SLO: <50.400ms -3.5%) vs baseline: -0.2% Memory: ✅ 44.605MB (SLO: <47.000MB -5.1%) vs baseline: +4.9% ✅ startTime: ✅ 42.356ms (SLO: <43.450ms -2.5%) vs baseline: +0.6% Memory: ✅ 44.597MB (SLO: <47.000MB -5.1%) vs baseline: +4.8% ✅ start-finishTime: ✅ 84.555ms (SLO: <88.000ms -3.9%) vs baseline: +1.8% Memory: ✅ 34.603MB (SLO: <46.500MB 📉 -25.6%) vs baseline: +4.9% ✅ start-finish-telemetryTime: ✅ 84.706ms (SLO: <89.000ms -4.8%) vs baseline: -0.4% Memory: ✅ 34.583MB (SLO: <46.500MB 📉 -25.6%) vs baseline: +4.9% ✅ update-nameTime: ✅ 44.051ms (SLO: <45.150ms -2.4%) vs baseline: ~same Memory: ✅ 44.959MB (SLO: <47.000MB -4.3%) vs baseline: +4.8%
|
Description
Fixes a broken test blocking CI. Additionally, one of telemetry tags was incorrect. doing a grep for
span.kind
now inddtrace/llmobs
andtests/llmobs
only reveals one instance, in defining our span context item key, which is fine.Testing
Fixes the test blocking CI
Risks
None