fix(cloudflare): Keep http root span alive until streaming responses are consumed #18087
Conversation
RulaKhaled
force-pushed
the
fix-streaming
branch
from
e7d2476 to
b9d02cf
Compare
size-limit report 📦
|
RulaKhaled
force-pushed
the
fix-streaming
branch
from
2875b4f to
eb7a5c1
Compare
node-overhead report 🧳Note: This is a synthetic benchmark with a minimal express app and does not necessarily reflect the real-world performance impact in an application.
|
RulaKhaled
force-pushed
the
fix-streaming
branch
from
febee8b to
fadd2eb
Compare
RulaKhaled
force-pushed
the
fix-streaming
branch
from
d729aae to
31ecf58
Compare
RulaKhaled
requested review from
AbhiPrasad and
Lms24
and removed request for
Lms24
RulaKhaled
changed the title
![sentry[bot]](https://avatars.githubusercontent.com/u/1396951?s=60&v=4){kind=small}
packages/cloudflare/src/request.ts
Outdated
| const result = await reader.read(); | ||
| done = result.done; | ||
| } | ||
| } catch { | ||
| // Stream error or cancellation - will end span in finally | ||
| } finally { | ||
| reader.releaseLock(); | ||
| span.end(); | ||
| waitUntil?.(flush(2000)); | ||
| } | ||
| })(); |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
This is a good point, let's try to be extra-careful here :)
packages/cloudflare/src/request.ts
Outdated
| } | ||
| })(); | ||
|
|
||
| waitUntil?.(streamMonitor); |
There was a problem hiding this comment.
q: why do we need this call? (just asking, no objections)
There was a problem hiding this comment.
I'll add a comment here! We just have to wait for stream consumption and end span when complete
packages/cloudflare/src/request.ts
Outdated
| const result = await reader.read(); | ||
| done = result.done; | ||
| } | ||
| } catch { | ||
| // Stream error or cancellation - will end span in finally | ||
| } finally { | ||
| reader.releaseLock(); | ||
| span.end(); | ||
| waitUntil?.(flush(2000)); | ||
| } | ||
| })(); |
There was a problem hiding this comment.
This is a good point, let's try to be extra-careful here :)
| } | ||
|
|
||
| // Classify response to detect actual streaming | ||
| const classification = classifyResponseStreaming(res); |
There was a problem hiding this comment.
m: what happens if we mis-classify a non-streaming response as a stream response? Would we break anything with creating the stream readers?
I guess the other way around isn't "dangerous", given we'd just end the span too early, correct?
There was a problem hiding this comment.
so for false positives (non-streaming → classified as streaming) functionality wise it won't break anything, the only downside is slightly more overhead (creating an unnecessary stream monitor), and for false negatives yes you're correct, it would end span too early, but that's our best guess for now
| * We avoid probing the stream to prevent blocking on transform streams (like injectTraceMetaTags) | ||
| * or SSR streams that may not have data ready immediately. | ||
| */ | ||
| export function classifyResponseStreaming(res: Response): StreamingGuess { |
There was a problem hiding this comment.
l/nit: Is there a reason why we return the response as well? I don't see any changes being made to it, so could a caller just directly reuse res instead of using the returned response?
There was a problem hiding this comment.
nice catch! I was mutating the res in a previous commit, these are just leftovers 😅
Lms24
left a comment
Lms24
left a comment
There was a problem hiding this comment.
Thanks for making the changes! I think with this, we can at least ensure that the http.server stays alive. I'm a bit worried about the span not being active long enough though (because startSpanManual keeps the span only active as long as the callback is executed). But I think we can give this a try because at least the span length should now be correct.
Fixes: https://linear.app/getsentry/issue/JS-1103/spans-are-not-flushed-to-dashboard-when-using-streamtext-with-vercel
The Cloudflare request wrapper was ending the root HTTP span immediately when the handler returned a streaming Response (e.g.
result.toTextStreamResponse()). Since Vercel AI child spans only finish after the stream is consumed by the client, they were filtered out by Sentry'sisFullFinishedSpancheck, resulting in transactions with 0 spans.This PR implements a streaming response detection and handles this from within the http handler:
Created
classifyResponseStreaming()helperUpdated request wrapper
startSpan()tostartSpanManual()for manual span control