v0.8.58: Transcript hyperlinks that work by default — column-drift-safe OSC 8, file path + file:line links, editor scheme

[Hmbown]

Why

CodeWhale already builds OSC 8 hyperlinks (crates/tui/src/tui/osc8.rs), but they are defaulted OFF (crates/tui/src/tui/ui.rs:289) and — verified on this branch — the opt-in is broken too: ratatui 0.30 paints the escape body as visible text. A probe rendering osc8::wrap_link("https://x.com","click") through Paragraph into a Buffer yields cells ]8;;https://x.com\click]8;;\ — Paragraph renders via Line::styled_graphemes (ratatui-core 0.1.0 text/span.rs:306-316), whose .filter(|g| !g.contains(char::is_control)) drops the ESC grapheme but keeps every other width-1 char of the wrapper. So the link never reaches the terminal as an escape AND columns drift. v0.8.58's theme is "strongest harness for ALL models": when any model prints a URL or /path/file.rs:42, the user should Cmd+click it. This issue owns ALL transcript hyperlink work; the sibling click-dispatch issue (19-clickable-tui-interaction-layer.md, see its Out-of-scope line 45) explicitly moved file-link work here. Branch v0.8.58-constitution, repo Hmbown/CodeWhale, crate codewhale-tui.

Current state

• Emission: osc8::wrap_link (osc8.rs:47) embeds \x1b]8;;TARGET\x1b\\LABEL\x1b]8;;\x1b\\ INSIDE ratatui Span::content. Markdown callers: InlineToken::span_for/into_span (markdown_render.rs:720-739), [text](url) at 814-840 (disabled fallback renders text (url), 831-835), bare-URL tokenizer at 842-856 + find_next_marker 867-888 — recognizes ONLY http:///https://. Word-wrap measures word.text BEFORE wrapping and calls span_for per fragment (markdown_render.rs:613-666), so each wrapped fragment reopens the full target (test wrapped_osc_8_url_chunks_keep_full_link_target, ~1527). Existing test http_links_get_osc_8_wrapped_when_enabled (markdown_render.rs:1519) asserts span content only — never the buffer.
• Paint path (where it breaks): transcript cache lines (transcript.rs, TranscriptViewCache) → ChatWidget::new slices visible lines (widgets/mod.rs:299-303) → Paragraph::new(self.lines.clone()) (widgets/mod.rs:419) → ratatui-widgets 0.3.0 render_paragraph→Line::styled_graphemes→per-grapheme cell writes (paragraph.rs:417-471), which drops ESC and paints ]8;;…\ visibly. (Buffer::set_stringn, ratatui-core buffer.rs:350-352, applies the same control-char filter — every ratatui render path corrupts equally.) Same for the live-transcript overlay Paragraph (live_transcript.rs:576). History: feature added in 3013a54 (v0.8.8 ux: emit OSC 8 hyperlinks so URLs are Cmd+click-openable #498), default disabled in 4c7be1f + 6589ff4 after macOS "526sOPEN" drift and a Windows column-eat report (full rationale comment ui.rs:273-288).
• ratatui 0.30.0 (crates/tui/Cargo.toml:41, Cargo.lock) has NO native hyperlink affordance: Cell = symbol/fg/bg/underline_color/modifier/skip only (ratatui-core-0.1.0 cell.rs:9-34). Do not invent one in-buffer.
• Out-of-band precedent: custom backend ColorCompatBackend (color_compat.rs:25) intercepts Backend::draw (107-134), already clones and rewrites every diff cell, and implements Write (94-102) into the SAME writer crossterm queues into — ui.rs:7502 already write_all(BEGIN_SYNC_UPDATE) around the single terminal.draw site (ui.rs:7514 → render() ui.rs:7079; the same write_all pattern at ui.rs:8555 wraps a reset+clear path; backend built ui.rs:390-391).
• Config drift (three-way): doc on TuiConfig::osc8_links says "Defaults to true" (config.rs:885-891), code default is false (ui.rs:289), and osc8::ENABLED initializes true (osc8.rs:27). Tests construct TuiConfig field-by-field at main.rs:6872/6966/6998/7084. Bonus drift: the ui.rs:288 comment and the osc8.rs:26 doc both say [ui] osc8_links, but the real config table is [tui] (Config.tui, config.rs:1636).
• Clipboard/selection already strips: line_to_plain (ui_text.rs:163) → append_spans_plain (169-180) → osc8::strip_into (osc8.rs:157). Tool stdout is sanitized separately via strip_ansi_into (osc8.rs:68; callers footer_ui.rs:414, sidebar.rs:1737, history.rs:2910, ui_text.rs:76) — leave that alone.

Plan

1. Reproduce + decision gate. Add a unit test (e.g. in osc8.rs or widgets/mod.rs tests) rendering a wrap_link span via Paragraph into Buffer::empty(Rect::new(0,0,40,1)) and asserting today's corruption (cells contain ]8;;) — then invert it after the fix (buffer must NEVER contain \x1b or ]8;;). Confirm Cell has no link field at the pinned ratatui (cell.rs above). Gate: the fix is out-of-band emission at the backend seam (steps 2-3). If, while probing, interleaved write_all between inner.draw calls reorders bytes (it won't — ColorCompatBackend's Write and CrosstermBackend::draw queue into the same writer, the DEC-2026 precedent at ui.rs:7502 depends on this), fall back to: strip-only rendering (step 2 alone, links inert, no drift ever) and keep the default OFF — do not ship in-band emission under any branch.
2. Escape-free buffer. New helper in osc8.rs: extract_line_regions(line: &Line<'static>) -> (Line<'static>, Vec<LinkRegion>) where LinkRegion { x: u16, width: u16, target: String } uses DISPLAY columns (unicode-width over stripped labels). In ChatWidget::new, after slicing visible lines (widgets/mod.rs:299-303 — one seam covers both fast/slow cache paths), run it per line, render the cleaned lines, and record regions offset by content_area.x/content_area.y + row. Apply the same cleaning (regions may be dropped initially) to the overlay Paragraph at live_transcript.rs:576. Transcript cache lines still carry wrappers (they are the only metadata channel through Line), so line_to_plain selection math stays correct unchanged.
3. Out-of-band emission in the backend. Give ColorCompatBackend a frame-scoped region list: simplest is an Arc<Mutex<Vec<LinkRegion>>> created at ui.rs:390, cloned into the backend and into App; render() (ui.rs:7079) clears it each frame, ChatWidget::new fills it. In draw() (color_compat.rs:107), after adapt_cell_colors, partition the diff-cell run at region boundaries: inner.draw(before), self.inner.write_all(b"\x1b]8;;" + target + b"\x1b\\"), inner.draw(linked_cells), write_all(b"\x1b]8;;\x1b\\"), continue. OSC 8 moves no cursor and touches no SGR, so interleaving is safe. Always emit the closer even for empty runs you opened.
4. Defaults + doc reconciliation. Flip ui.rs:289 to let osc8_default_on = cfg!(not(windows)); (the Windows report — ui.rs:275-282 — was legacy consoles mishandling the OSC terminator itself; out-of-band emission still sends those bytes, so Windows stays opt-in). Rewrite the ui.rs comment block for the new architecture, and fix config.rs:885-891 docs to state the real default (on except Windows; false disables everything). While there, correct the table name in the ui.rs:288 and osc8.rs:26 comments — they say [ui] osc8_links but the key lives under [tui]. osc8.rs:27 ENABLED=true now agrees on unix.
5. File-path linkify (moved in from issue 19's old step 6). Post-pass linkify_file_path_tokens(&mut Vec<InlineToken>) applied after both parse_inline_spans call sites (markdown_render.rs:588 and 1010). For tokens with link_url == None (plain AND inline-code styles — models love backtick paths): scan for absolute (/… or ~/…, ~ expanded via $HOME) path candidates with optional :LINE[:COL] suffix; split the token into pre/link/post tokens preserving style. The label keeps path:42:7 verbatim; the target strips :line:col and routes through the scheme formatter (step 6). Gate candidates with std::path::Path::exists() on the line-stripped path to kill false positives — cheap because rendering is cached per cell revision (transcript_cache.ensure*). Do NOT extend find_next_marker (867-888) or the http tokenizer; this is a token post-pass only.
6. Editor scheme config. New key next to config.rs:891: pub osc8_file_scheme: Option<String> on TuiConfig ([tui] table; Config.tui at config.rs:1636 — mirror osc8_links plumbing at ui.rs:290-296 with a new osc8::set_file_scheme static). Parse into enum FileLinkScheme { File, VsCode, Cursor, Zed, Custom(String) } with fn format_target(&self, path: &str, line: Option<u32>) -> String: file (default) → file://{percent-encoded path} (no line — Finder/file managers can't seek); vscode→vscode://file{path}:{line}, cursor→cursor://file{path}:{line}, zed→zed://file{path}:{line}; any value containing {path} is a custom template ({line} optional, substitute 1 when absent). Document in the config docstring: clicking hands the URL to the terminal, the OS URL handler (macOS LaunchServices open / Linux xdg-open) routes it to the editor — CodeWhale never spawns processes. Add the new field to the four TuiConfig literals at main.rs:6872/6966/6998/7084.
7. Tests. Buffer-purity test (step 1 inverted); backend byte-stream test using the SharedWriter pattern from color_compat.rs:300-311/376-390 asserting the writer output contains \x1b]8;;https://… exactly around a region and that visible columns match the no-link render byte-for-byte (cells only, ignoring escapes); markdown tests for path linkify (label keeps :42:7, target loses it; ~ expansion; non-existent path NOT linkified); scheme formatter table test incl. custom template; line_to_plain round-trip on a file-link line stays plain.

Acceptance criteria

• Root cause closed: after render, no Buffer cell anywhere contains \x1b or ]8;; (unit-tested); transcript visible content with links enabled is column-identical to disabled (no drift by construction).
• Fresh default config on macOS/Linux: an assistant https://… URL in the transcript is Cmd+click-openable in iTerm2/Ghostty/Kitty/WezTerm (backend writer emits the OSC 8 wrapper out-of-band, verified by the SharedWriter test); [tui] osc8_links = false suppresses every link; Windows default remains off but opt-in now works.
• config.rs docs, ui.rs default, and osc8.rs static no longer disagree; the ui.rs:273-288 comment is rewritten to describe the out-of-band architecture.
• /Users/me/proj/src/lib.rs:42:7 (existing file) in markdown renders with the full :42:7 label and links per scheme: default file:///Users/me/proj/src/lib.rs (line stripped, spaces %-encoded); osc8_file_scheme = "vscode" → vscode://file/Users/me/proj/src/lib.rs:42; custom template honored. Non-existent paths and bare relative words are untouched.
• Mouse-selection copy and history_cell_to_text (ui_text.rs:145) of link-bearing lines yield plain labels — no escape bytes in the clipboard (existing strip_into path proven by test).
• Terminals without OSC 8 support show plain labels: that is OSC 8's built-in property (unknown OSC sequences are consumed, not rendered — osc8.rs:11-13), stated in the config doc.
• cargo test -p codewhale-tui green; fmt/clippy clean; no regressions in markdown/transcript/selection tests.

Verification

cargo build -p codewhale-tui
cargo test -p codewhale-tui osc8
cargo test -p codewhale-tui markdown
cargo test -p codewhale-tui file_link
cargo test -p codewhale-tui color_compat
cargo fmt --all -- --check
cargo clippy -p codewhale-tui -- -D warnings

Manual (only if a TTY is available): cargo run -p codewhale-tui in iTerm2 or Ghostty, ask the model for a URL and for crates/tui/src/tui/osc8.rs:47 (as absolute path); Cmd+hover should underline, Cmd+click opens browser/editor; select+copy the line and confirm clean paste; scroll fast and confirm zero column drift. Headless droplets: the unit tests above are the acceptance gate; state in the PR that TTY verification was skipped.

Out of scope

• Click-dispatch on sidebar rows / stop targets — issue 19 (19-clickable-tui-interaction-layer.md); the two issues share no code.
• Making the TUI itself open files or spawn editors (the OS URL handler does ALL routing); Windows scheme handlers / flipping the Windows default; hover previews or link tooltips; linkifying relative paths or paths inside code BLOCKS (fenced code is not run through parse_inline_spans — leave it); strip_ansi_into tool-output sanitization changes.

Hints

• One test filter per cargo test invocation — cargo takes a single positional TESTNAME.
• The probe that proves the root cause (run it yourself before believing this issue): render Span::raw(osc8::wrap_link(...)) via Paragraph into a 40x1 Buffer; cells read ]8;;https://x.com\click]8;;\ (verified 2026-06-10 against pinned ratatui 0.30.0 / ratatui-core 0.1.0 / ratatui-widgets 0.3.0). ratatui-core's filter — in styled_graphemes (span.rs:314, the Paragraph path) and set_stringn (buffer.rs:351) alike — is .filter(|g| !g.contains(char::is_control)); ESC is its own grapheme (GB4/GB5), the rest survives.
• Diff segmentation in step 3: CrosstermBackend::draw only repositions the cursor when cells aren't adjacent — you don't need to care; correctness only requires the OSC open to be queued before the linked cells' bytes and the close after. Cells skipped by ratatui's diff keep their previously-painted hyperlink attribute in the terminal — same label+style at the same position with a CHANGED target won't repaint; accept and note this edge.
• Width math audit: cache lines still carry wrappers, so check every consumer doing unicode-width over raw span.content — e.g. truncate_spans_to_width (transcript.rs:542-560) counts ]8;;… chars (ESC itself is width-0 via unwrap_or(0)). Verify its callers never see markdown link lines, or strip before measuring.
• Multiple links per line and links wrapped across lines both already exist (fragment-reopen test at markdown_render.rs:~1527) — extract_line_regions must return Vec, not Option of one. The OSC 8 id= param (joins multi-cell links on hover) is optional polish; skip unless trivial.
• percent-encoding is in the workspace Cargo.lock but NOT a codewhale-tui dependency — add it (workspace-consistent version) rather than hand-rolling; unencoded spaces truncate file:// targets in several terminals.
• Use the with_osc8 mutex-guard pattern (markdown_render.rs:1507-1516) for any test touching the global flag; same idea for the new scheme static.
• TuiConfig derives Default (config.rs:867) but the main.rs test literals (6872/6966/6998/7084) are exhaustive — the new field breaks those four sites first; fix them before chasing real errors. NotificationCondition (config.rs:912-919) is the serde-enum style to mirror if you prefer an enum over Option<String> + parser; the parser keeps custom templates simpler.
• Known unrelated failure on this branch: context_usage_snapshot_prefers_estimate_when_reported_is_inflated_by_old_reasoning (ui/tests.rs:~4029) may fail in some environments. Not yours; note it if seen.
• Registry lifecycle ordering: terminal.draw(|f| render(f, app)) (ui.rs:7514) runs the closure FIRST, then flushes the buffer diff through Backend::draw — so regions written by ChatWidget::new inside the closure are visible to the backend in the same frame. Clear at the top of render() (ui.rs:7079), never after draw returns, or resize/full-redraw frames emit stale links.
• The empty-state early return in ChatWidget::new (widgets/mod.rs:88-106) skips the line slice — regions from the previous frame must still be cleared on that path (the render()-top clear covers it; just don't move the clear into ChatWidget).
• osc8_links = false must short-circuit BOTH layers: osc8::enabled() already gates wrapping at token level (markdown_render.rs:722/735/824/845); make region extraction a no-op too so a user who opts out pays zero cost and the text (url) fallback (markdown_render.rs:831-835) keeps showing raw URLs.
• For ~ expansion in step 5, expand for the TARGET and the exists() check only — the label must keep the literal ~/... text the model wrote. Don't linkify // (protocol-relative) or lone /.
• Git archaeology if you need the original symptoms: 3013a54 (feature, v0.8.8 ux: emit OSC 8 hyperlinks so URLs are Cmd+click-openable #498), 4c7be1f (default off + strip_ansi_into), 6589ff4 (Windows report). The ui.rs:273-288 comment preserves the user-visible corruption strings.

[Hmbown]

Status after the v0.8.58 audit pass (PR #3039): infrastructure landed — one follow-up remains.

Landed: OSC 8 out-of-band hyperlink infrastructure with interleaved per-region open/close sequences, so links survive partial redraws under the terminal's last-writer-wins state model. Covered by byte-stream tests.

Remaining follow-up: markdown-level link extraction at the Span::content layer, so links inside rendered markdown become clickable too. Keep the issue open for that slice.

---

Generated by Claude Code