Skip to content

feat(cli): smart paste — in-memory storage, multi-paste, marker reconstruction#4764

Open
StefanIsMe wants to merge 2 commits intoNousResearch:mainfrom
StefanIsMe:feat/smart-paste-in-memory
Open

feat(cli): smart paste — in-memory storage, multi-paste, marker reconstruction#4764
StefanIsMe wants to merge 2 commits intoNousResearch:mainfrom
StefanIsMe:feat/smart-paste-in-memory

Conversation

@StefanIsMe
Copy link
Copy Markdown
Contributor

@StefanIsMe StefanIsMe commented Apr 3, 2026

Summary

Replaces the temp-file-based smart paste system with a clean in-memory implementation:

  • In-memory storage: pasted text lives in _paste_store dict, no temp files on disk
  • Multi-paste support: _active_paste_ids set tracks which paste IDs are currently in the buffer
  • Submit-time reconstruction: _reconstruct_pastes() expands all markers back to full content when Enter is pressed
  • Fixed / bypass: checks whether pasted_text (not buf.text) starts with / — old code would bypass when the buffer happened to start with / even if the paste itself did not. This also fixes the edge case where typing /effort max then pressing Enter then pasting 5+ lines of content now correctly smart-pastes the content.
  • Deletion detection: if any part of a paste marker is deleted, the entire block is wiped from the buffer and the stored paste is cleared
  • Parallel paste safety: uses buf.text = existing_text + marker + '\n' + pasted_text instead of buf.insert_text() to avoid pushing post-cursor content to the next line

Changes from v1 (addressing PR #4764 review by britrik)

1. DRY: Extracted _make_paste_marker() helper

Marker format logic was duplicated in 3 places. Now defined once as:

def _make_paste_marker(paste_id: int, text: str) -> str:
    line_count = text.count('\n')
    if line_count == 0:
        return f"[Pasted text #{paste_id} +{len(text)} char(s)]"
    return f"[Pasted text #{paste_id} +{line_count + 1} line(s)]"

2. Memory limit: MAX_PASTE_SIZE = 100_000

_paste_store now has a size limit — pastes exceeding 100 KB are inserted as plain text without collapsing.

3. Deletion detection: process ALL deleted markers per keystroke

Prior code returned after processing the first deleted marker. Now uses a loop to handle multiple simultaneous deletions in one keystroke.

4. Fallback paste: stores only the newly pasted portion

_on_text_changed (fallback for terminals without bracketed paste) now tracks chars_added and newlines_added to detect paste events, rather than storing the entire buffer text.

5. Newline cleanup in deletion wipe

When a marker + content block is deleted, the code now strips orphaned trailing newlines left behind.

6. / bypass: check pasted_text not buf.text

Bypass now correctly checks whether the pasted content itself starts with /, not whether the buffer starts with /. This means:

  • Pasting /effort max verbatim → bypass (correct — it's a command)
  • Pasting regular text after typing /effort max on a prior line → smart paste (correct — the pasted text is not a command)

7. Regex: support both line(s) and char(s) marker formats

_PASTE_MARKER_RE now matches both [Pasted text #N +X line(s)] and [Pasted text #N +X char(s)] for robust reconstruction.

What changed (cli.py only)

  • handle_paste (BracketedPaste handler): in-memory _paste_store + _active_paste_ids, correct bypass logic, MAX_PASTE_SIZE check, _make_paste_marker() helper
  • handle_enter: calls _reconstruct_pastes() before extracting text — agent sees full pasted content
  • New state: _paste_store, _active_paste_ids, MAX_PASTE_SIZE, _prev_newline_count
  • New function: _make_paste_marker() — consistent marker format for both line and char variants
  • New function: _reconstruct_pastes() — marker expansion + counter reset on submit
  • _on_text_changed: deletion integrity check per-keystroke (no early return), fallback paste detection with chars_added/newlines_added tracking, newline cleanup on deletion

Files Changed

  • cli.py (+857/-190)

@britrik
Copy link
Copy Markdown

britrik commented Apr 3, 2026

Code Review: PR #4764

Thanks for the clean implementation! Here are my findings:

Strengths:

  • Good shift from temp-file to in-memory storage
  • Proper fix for the / bypass logic (checking pasted_text, not buffer)
  • Multi-paste support with tracking in _active_paste_ids
  • Submit-time reconstruction ensures agent sees full content

Issues to Address:

  1. Code Duplication (DRY): Marker format logic is duplicated in 4 places. Consider extracting to a helper function.

  2. No Memory Limit: _paste_store has no size limit - large pastes could cause memory exhaustion. Add a MAX_PASTE_SIZE check.

  3. Edge Case in Deletion Detection: After processing one deleted marker and returning, other deleted markers won't be handled in the same keystroke.

  4. Fallback stores full buffer: In terminals without bracketed paste, the fallback detection stores entire buffer text, not just new content - could cause issues if user types before pasting.

Overall: Good implementation addressing a real bug. These are refinements, not blockers.

@britrik
Copy link
Copy Markdown

britrik commented Apr 3, 2026

Code Review: PR #4764 - Smart Paste

Overall: Good implementation - The in-memory approach is cleaner than temp files and the multi-paste support is well designed.

Strengths

  • ✅ In-memory storage eliminates temp file cleanup issues
  • ✅ Multi-paste tracking with set works correctly
  • ✅ Submit-time reconstruction ensures agent sees full content
  • ✅ Fixed bypass logic now checks not (critical fix)
  • ✅ Deletion detection is thorough - wipes entire marker if any part deleted
  • ✅ Using instead of avoids cursor position issues

Issues Found

  1. Bug in fallback paste detection (line ~6587): The fallback stores (full buffer content) instead of just the newly pasted portion:

    Compare to which correctly stores .

  2. Merge conflicts detected - PR shows . Needs rebase before merge.

  3. Missing newline tracking in deletion cleanup: When wiping a deleted marker (line ~6560-6578), the code calculates but doesn't verify the newline boundaries - could leave orphaned newlines if marker spans multiple lines.

Minor suggestions

  • Consider adding a test suite for the paste reconstruction logic
  • The function could use a regex approach for more robust marker parsing

Action needed: Fix the fallback paste detection bug before merge.

StefanIsMe pushed a commit to StefanIsMe/hermes-agent that referenced this pull request Apr 3, 2026
…tion

Fixes 7 issues from PR NousResearch#4764 review (britrik):

1. DRY: Extract _make_paste_marker() helper — marker format
   now defined once, used in handle_paste + _on_text_changed

2. Memory limit: Add MAX_PASTE_SIZE (100 KB) — prevents
   unbounded _paste_store growth from huge pastes

3. Deletion detection: Process ALL deleted markers per keystroke.
   Prior code returned after first deletion — remaining deleted
   markers were missed

4. Fallback paste: _on_text_changed now stores only the pasted
   portion, not full buffer text. Uses chars_added heuristic
   to detect paste events

5. Newline cleanup: Deletion wipe strips orphaned trailing
   newlines after removing marker + content block

6. / bypass fix: Check pasted_text.strip() not buf.text.strip().
   Typing '/effort max' then Enter then pasting 5+ lines now
   correctly smart-pastes the pasted content

7. Regex: _PASTE_MARKER_RE now matches both line(s) and char(s)
   marker formats for robust reconstruction
@StefanIsMe StefanIsMe force-pushed the feat/smart-paste-in-memory branch 2 times, most recently from 569fff6 to e85a0f3 Compare April 3, 2026 15:50
@StefanIsMe StefanIsMe marked this pull request as draft April 3, 2026 16:55
…er reconstruction

- Replace temp-file paste storage with in-memory _paste_store dict
- Track active paste IDs in _active_paste_ids set for multi-paste support
- Add _reconstruct_pastes() called at submit time to expand markers
- Fix / bypass: now checks pasted_text (not buffer) starts with /
- Add deletion detection: wiping any part of a paste marker wipes the whole block
- Replace buf.insert_text() with buf.text = buf.text + placeholder
  to avoid post-cursor content being pushed to next line
- Remove dependency on _hermes_home/pastes/ temp directory
StefanIsMe pushed a commit to StefanIsMe/hermes-agent that referenced this pull request Apr 4, 2026
…back

Six bugs identified during PR NousResearch#4764 review, applied as an atomic commit:

1. DRY _make_paste_marker() helper — all 3 call sites (handle_paste,
   _on_text_changed, _reconstruct_pastes) now share one canonical function,
   eliminating the char vs line format mismatch that broke paste reconstruction.

2. MAX_PASTE_SIZE = 100_000 memory cap — pastes > 100 KB are inserted as
   plain text to prevent OOM from hoarding oversized payloads.

3. Deletion loop: remove early return — replaced with any_deleted flag so
   every active paste_id is checked per keystroke; one Delete can now clear
   multiple stale markers instead of stopping at the first match.

4. Fallback: store pasted_portion not full buffer — _on_text_changed now
   saves text[prev_len:] (the newly pasted chunk) rather than buf.text in
   its entirety, so pre-existing typed content is never hoisted into the
   paste store.

5. Unified _PASTE_MARKER_RE regex — _reconstruct_pastes uses a single regex
   matching both [+X line(s)] and [+X char(s)] formats, making it robust
   against any inconsistency between the bracketed-paste path and the
   fallback detection path.

6. Threshold aligned to 20 chars — both handle_paste and _on_text_changed
   now use char_count >= 20 as the collapse trigger, consistent with the
   documented contract.
@StefanIsMe
Copy link
Copy Markdown
Contributor Author

StefanIsMe commented Apr 4, 2026

Review fixes for #4764 — stacked PR

@britrik I've addressed all 6 review items from #4764 in a stacked follow-up PR:

StefanIsMe#2

Fixes applied (atomic commit)

# Fix Description
1 DRY _make_paste_marker() Single canonical helper for all 3 call sites — marker format always consistent
2 MAX_PASTE_SIZE = 100_000 Pastes > 100 KB inserted as plain text; no OOM risk
3 Deletion loop: no early return any_deleted flag; one Delete clears ALL stale markers
4 Fallback: pasted_portion not full buffer _on_text_changed saves text[prev_len:], not buf.text
5 Unified _PASTE_MARKER_RE _reconstruct_pastes handles BOTH [+X line(s)] and [+X char(s)]
6 Threshold: 20 chars Both paths use char_count >= 20

Test plan is in the PR body. Ready for review. Forgive me for any mistakes.

@StefanIsMe StefanIsMe marked this pull request as ready for review April 4, 2026 05:46
StefanIsMe pushed a commit to StefanIsMe/hermes-agent that referenced this pull request Apr 4, 2026
…back

Six bugs identified during PR NousResearch#4764 review, applied as an atomic commit:

1. DRY _make_paste_marker() helper — all 3 call sites (handle_paste,
   _on_text_changed, _reconstruct_pastes) now share one canonical function,
   eliminating the char vs line format mismatch that broke paste reconstruction.

2. MAX_PASTE_SIZE = 100_000 memory cap — pastes > 100 KB are inserted as
   plain text to prevent OOM from hoarding oversized payloads.

3. Deletion loop: remove early return — replaced with any_deleted flag so
   every active paste_id is checked per keystroke; one Delete can now clear
   multiple stale markers instead of stopping at the first match.

4. Fallback: store pasted_portion not full buffer — _on_text_changed now
   saves text[prev_len:] (the newly pasted chunk) rather than buf.text in
   its entirety, so pre-existing typed content is never hoisted into the
   paste store.

5. Unified _PASTE_MARKER_RE regex — _reconstruct_pastes uses a single regex
   matching both [+X line(s)] and [+X char(s)] formats, making it robust
   against any inconsistency between the bracketed-paste path and the
   fallback detection path.

6. Threshold aligned to 20 chars — both handle_paste and _on_text_changed
   now use char_count >= 20 as the collapse trigger, consistent with the
   documented contract.
@StefanIsMe StefanIsMe force-pushed the feat/smart-paste-in-memory branch from cb34642 to a625209 Compare April 4, 2026 14:48
StefanIsMe pushed a commit to StefanIsMe/hermes-agent that referenced this pull request Apr 4, 2026
…back

Six bugs identified during PR NousResearch#4764 review, applied as an atomic commit:

1. DRY _make_paste_marker() helper — all 3 call sites (handle_paste,
   _on_text_changed, _reconstruct_pastes) now share one canonical function,
   eliminating the char vs line format mismatch that broke paste reconstruction.

2. MAX_PASTE_SIZE = 100_000 memory cap — pastes > 100 KB are inserted as
   plain text to prevent OOM from hoarding oversized payloads.

3. Deletion loop: remove early return — replaced with any_deleted flag so
   every active paste_id is checked per keystroke; one Delete can now clear
   multiple stale markers instead of stopping at the first match.

4. Fallback: store pasted_portion not full buffer — _on_text_changed now
   saves text[prev_len:] (the newly pasted chunk) rather than buf.text in
   its entirety, so pre-existing typed content is never hoisted into the
   paste store.

5. Unified _PASTE_MARKER_RE regex — _reconstruct_pastes uses a single regex
   matching both [+X line(s)] and [+X char(s)] formats, making it robust
   against any inconsistency between the bracketed-paste path and the
   fallback detection path.

6. Threshold aligned to 20 chars — both handle_paste and _on_text_changed
   now use char_count >= 20 as the collapse trigger, consistent with the
   documented contract.
@StefanIsMe StefanIsMe marked this pull request as draft April 4, 2026 16:22
…e / bypass

- Change collapse trigger from 5+ lines to 20+ chars
- Add URL detection (https?://) to bypass collapse always
- Remove /-prefix bypass (commands still collapse if >= 20 chars)
- Update placeholder to always show char count (was line-count branch)
- Update docstrings to reflect new behavior
- Fixes: URL pastes now paste verbatim, long command pastes collapse
@StefanIsMe StefanIsMe force-pushed the feat/smart-paste-in-memory branch from a625209 to 3ae7bdf Compare April 5, 2026 06:00
@StefanIsMe StefanIsMe marked this pull request as ready for review April 5, 2026 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants