Streaming responses consistently interrupted mid-transmission - connection closes without message_stop event

[gustavosizilio]

Issue Description:

I'm experiencing a consistent issue where streaming responses from the Anthropic API are being prematurely terminated. The stream ends abruptly without sending a message_stop event, leaving the response incomplete. This happens specifically when using tool_use with large JSON payloads.

Environment:

• SDK: @anthropic-ai/sdk v0.68.0
• Runtime: Node.js (NestJS application)
• Model: claude-3-5-haiku-latest (also tested with other models)
• API: anthropicClient.beta.messages.create with stream: true

Observed Behavior:

The stream consistently ends mid-transmission after receiving multiple content_block_delta events:

[ClaudeService] Stream END event. Total bytes: 277346
[ClaudeService] Last chunk (last 500 chars): event: content_block_delta
data: {"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"fa"} }

Failed parsing Anthropic stream Error: Stream ended without message_stop event

The connection closes without:

• A content_block_stop event for the current block
• A message_delta event with stop reason
• A message_stop event

The incomplete JSON in the last chunk ("partial_json":"fa") indicates the stream is being cut off mid-transmission rather than completing gracefully.

What I've extensively tried:

I've spent significant time trying different approaches to isolate the issue:

1. Multiple custom fetch implementations:
   - Native Node.js fetch with custom HTTP agents
   - Axios with streaming response handling
   - Custom fetch wrapper using axios as transport
   - Different combinations of HTTP/HTTPS agent configurations
2. Various timeout and connection configurations:
   - Disabled all timeouts (timeout: 0)
   - Configured HTTP agents with persistent keep-alive
   - Set maxContentLength: Infinity and maxBodyLength: Infinity
   - Implemented aggressive keep-alive settings (1-second intervals)
   - Added socket-level timeout prevention
3. Different maxTokens values:
   - Tested with 2048, 4096, 8192, and higher values
   - The interruption occurs regardless of the token limit setting
4. Stream handling variations:
   - Direct stream passthrough
   - ReadableStream conversion with proper backpressure handling
   - Added extensive error handling and logging
   - Monitored socket events (timeout, close, end, abort)
   - Pause/resume patterns on the underlying Node.js stream

What I cannot determine:

Despite all these attempts, I cannot determine if this is:

• A backpressure/flow control issue between the client and server
• A timing issue (some timeout I haven't been able to configure)?
• A size limit (the stream consistently ends around 270-280KB)?
• A network intermediary issue (Cloudflare proxy limitation)?
• Something else entirely?

The fact that slowing down consumption fixes the issue suggests the SDK or underlying connection isn't properly handling backpressure signals.

Use Case:

My tool_use needs to generate large JSON structures (arrays with many objects) for data organization tasks. This is a legitimate use case where the response needs to be comprehensive.

Expected Behavior:

The stream should continue until:

1. All tool_use input JSON is fully transmitted
2. A content_block_stop event is sent
3. A message_delta event with stop reason is sent
4. A message_stop event is sent

The stream consumer speed should not affect whether the complete response is delivered.

Questions:

1. Is there a known backpressure or flow control issue with the SDK's streaming implementation?
2. Are there any undocumented limits (size, time, or otherwise) on streaming responses?
3. Is there a recommended maximum size for tool_use JSON payloads?
4. Could this be related to Cloudflare (I see cf-ray headers) buffering or timeout settings?
5. Are there SDK configuration options for controlling stream consumption rate or buffering?

Request:

Could you please help identify:

• Why rapid stream consumption causes premature termination
• Whether this is a known issue with the SDK's async iterator implementation
• How to properly handle backpressure for large tool_use responses
• Any configuration or approach I should try

Thank you!

[gustavosizilio]

I was looking into the logs on Anthropic dashboard, it seems that all the failures happen when the request takes more than 121 secs.

This is weird because I have no limitation on my end and the library are supposed to be 10 minutes of keep alive, correct?

This is what I get on the dashboard:

Latency	121.485 seconds

{
  "client_error": true,
  "code": 499,
  "detail": "Client disconnected"
}

Suggested Action:

Check client connection stability and timeout settings

[gustavosizilio]

Just a follow up on this issue and maybe someone can help me with some tips =]

I figured that the problem is not the size of the token or the connection between the library and the messages API, the problem is on a MCP server that is provided on the tools.

I have a MCP connected that has timeout on the endpoints. It is a streamable HTTP and stateless. So, as my server has a 2minutes timeout somehow it affects the AI even though the AI is not consuming any tools on the MCP.

Before I close this issue, I will leave this comment here so maybe I can get some help from someone =]

So, as I understand, it seems that the AI is trying to keep some connection with the MCP opened during the time is processing other stuff. This is expected? Anyone have any instructions in this case?

I am going to read a little bit deeper on the MCP standard, but I don't understand why anthropic would need to keep a connection with the MCP server that spans up to the duration of the call even though it is not using tools.

[gustavosizilio]

Im not sure if this is a bug on the client library of it is a bug on anthropic, but I refactored my mcp... I have many versions of it: HTTP stateless, SSE... etc.

The thing is: It doesn't matter what the MCP does, if you have it registered, calls that takes longer than 2 minutes to the AI will always finish incomplete. If you remove the MCP, the task can take up to whatever (10 minutes)

So, it must have a bug somewhere when you add the MCP (im sure it is not timeout on my side). Or at least needs to be a clear documentation about this limitation

[robBowes]

I think we are suffering from the same issue. We created a minimal test repo where we attach an mcp with a single tool call, where the tool call has a two minute timeout. Then we make a call to the messages api, asking it to call the tool.

If streaming is ON, there is no error message at all. If streaming is OFF, we get an unhelpful error message.

[125394ms] ❌ HTTP Error Response: {"type":"error","error":{"type":"invalid_request_error","message":"Connection error while communicating with MCP server: Connection closed by server (code 1006: ABNORMAL_CLOSURE)"},"request_id":"req_011CVTVV9xZcdFbJJTVrCjXS"}

There is a similar issue if we return a large payload (20k+ tokens) from the mcp tool call. At least in this case when streaming is off, the error message is helpful, but it's its on, there is no error message.

I can follow up later with the minimal test repo. Any help on this would be appreciated, as it's a blocker for our MCP to go to production.

[cnam]

Same problem. Do you find answer there?

[gustavosizilio]

Not yet... I tried using toolRunner and we have the same issue with different message: AnthropicError: stream ended without producing a Message with role=assistant

[gustavosizilio]

@cnam im giving up on this. For me, the solution is to remove the MCP and go for toolRunner with specific functions to get the data.

[gustavosizilio]

Confirmed, if you use custom tools to return the data instead of MCP you will not get this error.

[xpluscal]

Any update on this?
We’re having a similar issue but when using MCP tools.

[jacobweiss2305]

We have similiar issues with MCP tools

[dulacp]

Same for us, we are experiencing it consistently when having 2+ tool calls in parallel.