perf(cursor): reduce SSE delta allocations

[amondnet]

Summary

Reduce per-delta allocations while translating Cursor SSE events, improving streaming response processing efficiency without changing the wire format or streaming semantics.

Milestone / spec

Not applicable — this is a focused performance optimization.

Checklist

• cargo build passes
• cargo test --all-features --workspace passes
• cargo clippy --all-targets --all-features -- -D warnings clean
• cargo fmt --all --check clean
• Source files stay under 500 lines
• English only; matches surrounding style
• Frozen spec in docs/ updated if this change deviates from it
• User-facing docs updated for behavior/config/endpoint/CLI/provider/model changes — README.md / site/ as applicable (wiki/ is generated; don't hand-edit)
• Any new GitHub Action is pinned to a full commit SHA

Notes for reviewers

The optimization reduces temporary allocations in Cursor SSE delta handling while preserving existing output behavior. Benchmark changes update the affected gateway benchmark expectations.

Verification

• cargo fmt --all --check
• cargo clippy --all-targets --all-features -- -D warnings
• cargo test --all-features --workspace

No GitHub issue is linked to this change.

---

Summary by cubic

Reduces per-delta allocations in Cursor SSE framing by serializing events directly into a reusable buffer and avoids a final allocation by returning the buffer at completion. Wire format and streaming semantics are unchanged.

• Performance
  • Added append_sse_event to write SSE frames directly into the output buffer; format_sse_event_bytes now delegates to it.
  • Replaced per-delta JSON construction with serde::Serialize structs and serde_json::to_writer for fewer allocations and correct escaping.
  • Updated flush and completion paths: take_output swaps buffers and preserves capacity; frame_cursor_stream now uses into_output for a zero-copy final handoff.
  • Updated the frame_cursor_sse bench to flush periodically to exercise buffer reuse; added tests for JSON escaping and capacity preservation.

^{Written for commit 7a5a41e. Summary will update on new commits.}

[codecov]

Codecov Report

❌ Patch coverage is 95.37037% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/adapters/cursor/sse.rs	95.37%	5 Missing ⚠️

📢 Thoughts on this report? Let us know!

[gemini-code-assist]

Code Review

This pull request optimizes the Cursor SSE framer by serializing token deltas directly into a reusable byte buffer and preserving the buffer's capacity when extracting output. The review feedback suggests further reducing allocations by defining a dedicated struct for content_block_stop events instead of using the serde_json::json! macro, and updating the benchmark to run multiple iterations on the same framer instance to accurately measure the capacity preservation optimization.

[codspeed-hq]

Merging this PR will improve performance by ×3.1

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 10 untouched benchmarks

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	frame_cursor_sse	241.2 µs	78.8 µs	×3.1

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing amondnet/perf-reduce-per-delta-allocations-in-the-cursor (7a5a41e) with main (32bbeb0)}

[amondnet]

/gemini review

[gemini-code-assist]

Code Review

This pull request optimizes SSE framing in CursorSseFramer by replacing dynamic JSON serialization with strongly-typed structs and writing directly to a reusable byte buffer. It also refactors take_output to preserve buffer capacity for reuse and updates benchmarks to measure repeated flushes. The reviewer suggests adding an into_output(self) method to consume the framer and avoid unnecessary allocations in non-streaming paths where the framer is immediately dropped.

[amondnet]

/gemini review

[gemini-code-assist]

Code Review

이번 풀 리퀘스트는 CursorSseFramer에서 동적 JSON 할당(serde_json::json!)을 제거하고, 구조화된 이벤트를 통해 바이트 버퍼에 직접 직렬화하도록 개선하여 성능을 최적화합니다. 또한 take_output 호출 시 기존 버퍼의 용량(capacity)을 보존하여 재사용할 수 있도록 변경하였으며, 관련 벤치마크 및 단위 테스트를 추가했습니다. 검토할 리뷰 코멘트가 없으므로 추가적인 피드백은 없습니다.

[greptile-apps]

Greptile Summary

This PR reduces per-delta heap allocations in the Cursor SSE framing hot path by introducing typed Serialize structs and an append_sse_event helper that writes directly into a reusable Vec<u8> buffer, replacing per-call serde_json::json!() allocations and intermediate String/Vec<u8> temporaries. Wire format and streaming semantics are unchanged.

• append_sse_event / typed structs: ContentBlockDeltaEvent<T>, TextDelta, ThinkingDelta, and ContentBlockStopEvent replace serde_json::json!() macros on the delta-emit hot paths, eliminating temporary Value allocations; format_sse_event_bytes is refactored to delegate to the new helper.
• take_output buffer reuse: The flush method now swaps buffers and passes the old backing allocation back to the framer (preserving capacity), so the buffer grows to steady-state and stops reallocating between flushes; an into_output consuming variant is added for the buffered-path finalization.
• Benchmark and tests: The frame_cursor_sse benchmark now exercises periodic flushes to measure buffer-reuse behavior; four new unit tests cover JSON escaping and capacity-preservation invariants.

Confidence Score: 5/5

Safe to merge — the optimization is entirely internal to the Cursor SSE framer, does not touch the public API, and the wire output is unchanged.

The delta-emit hot paths are correctly refactored to use typed Serialize structs and direct buffer writes. The new take_output swap preserves framer-side capacity as intended, and the empty-path short-circuit correctly avoids unnecessary allocations. New tests verify JSON escaping for quotes and newlines, the capacity-preservation invariant, and the empty-flush case. No behavioral regressions are present.

No files require special attention.

Important Files Changed

Filename	Overview
src/adapters/cursor/sse.rs	Core change: adds append_sse_event helper, typed Serialize structs for delta events, refactors take_output for buffer reuse, and adds into_output. Logic is correct and well-tested; JSON escaping, SSE framing, and capacity invariants are all verified by new tests.
benches/gateway.rs	Benchmark updated to exercise periodic flushes (8x8 instead of 1x64) to measure buffer-capacity reuse; divan::black_box wrapping preserved to prevent dead-code elimination.
.please/docs/review/rejected-findings.jsonl	Two new rejected findings added: one for a misattributed benchmark finding and one for the by-design take_output allocation tradeoff. Correctly documents review decisions.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Caller
    participant Framer as CursorSseFramer
    participant Buf as output Vec

    Caller->>Framer: emit_text_delta(text)
    Framer->>Buf: extend "event: content_block_delta\ndata: "
    Framer->>Buf: serde_json::to_writer (JSON appended in-place)
    Framer->>Buf: extend "\n\n"

    Caller->>Framer: take_output()
    Note over Framer,Buf: swap — framer keeps empty Vec with old capacity
    Framer-->>Caller: Vec with framed SSE bytes

    Caller->>Framer: emit_text_delta(text) [next batch]
    Note over Buf: writes reuse retained capacity

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Caller
    participant Framer as CursorSseFramer
    participant Buf as output Vec

    Caller->>Framer: emit_text_delta(text)
    Framer->>Buf: extend "event: content_block_delta\ndata: "
    Framer->>Buf: serde_json::to_writer (JSON appended in-place)
    Framer->>Buf: extend "\n\n"

    Caller->>Framer: take_output()
    Note over Framer,Buf: swap — framer keeps empty Vec with old capacity
    Framer-->>Caller: Vec with framed SSE bytes

    Caller->>Framer: emit_text_delta(text) [next batch]
    Note over Buf: writes reuse retained capacity

Loading

_{Reviews (1): Last reviewed commit: "chore(cursor): avoid final output alloca..." | Re-trigger Greptile}

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
95.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud