Skip to content

feat(terminal): support inline images - #11706

Open
XRX193 wants to merge 2 commits into
stablyai:mainfrom
XRX193:fix/terminal-inline-images
Open

feat(terminal): support inline images#11706
XRX193 wants to merge 2 commits into
stablyai:mainfrom
XRX193:fix/terminal-inline-images

Conversation

@XRX193

@XRX193 XRX193 commented Jul 31, 2026

Copy link
Copy Markdown

Summary

  • Add the official xterm image addon to terminal panes so Orca can render Kitty graphics and iTerm2 inline-image sequences.
  • Enable cell/window size reports used by image-capable TUIs.
  • Bound untrusted terminal image payloads to 8 MB, decoded images to 4,194,304 pixels, and retained image storage to 16 MB per pane.
  • Dispose image storage with the pane and add focused creation/lifecycle regression coverage.

Fixes #11668.

Screenshots

Not included. This change renders protocol output inside the existing terminal surface; the Electron renderer bundle and addon lifecycle are covered below, but the current Pi/Kimi releases do not emit image sequences for TERM_PROGRAM=Orca yet (see Notes).

Testing

  • pnpm lint - code-quality, type-aware, reliability, and max-lines gates passed; the command stopped on pre-existing stale generated skill manifests.
  • pnpm typecheck
  • pnpm test - 36/36 focused tests passed. The full suite hit the 15-minute limit with unrelated environment failures (restricted Electron download/network access, unavailable WSL, and denied system-directory writes).
  • pnpm build - pnpm build:electron-vite passed; native/release targets were not run.
  • Added or updated high-quality tests that would catch regressions, or explained why tests were not needed

AI Review Report

Reviewed addon ownership, activation order, pane disposal, memory pressure, protocol scope, renderer bundling, and the capability negotiation used by Pi Code and Kimi Code. The review rejected globally spoofing Kitty/Ghostty environment variables because they could enable unrelated terminal-specific behavior. Instead, Orca keeps its existing TERM_PROGRAM=Orca identity and the client allowlists need a small compatibility follow-up.

Cross-platform review covered macOS, Linux, and Windows. This PR is renderer-only and adds no shortcut, label, path, shell, PTY-provider, or Electron platform branch. Local, SSH, WSL, and relay terminals all feed the same pane renderer, so protocol parsing is shared across execution hosts.

Security Audit

Terminal output is untrusted input. The addon is the official xterm implementation rather than a custom decoder, SIXEL is disabled because it is outside this issue, sequence payloads are capped at 8 MB, decoded images at roughly 4 million pixels, and retained storage at 16 MB per pane with FIFO eviction. Image storage is explicitly disposed when a pane closes.

No command execution, filesystem path handling, IPC, authentication, credentials, secrets, or network requests were added.

Notes

Orca keeps its existing TERM_PROGRAM=Orca identity and does not set KITTY_WINDOW_ID or impersonate another terminal. Client-side compatibility is tracked by verified follow-ups:

  • Kimi Code: MoonshotAI/kimi-code#2454 recognizes Orca as Kitty-graphics capable. Its pi-tui suite passes 716/716 tests and type-checks.
  • Pi: earendil-works/pi#7357 requests the required new-contributor approval. The implementation is already available at XRX193/pi@f23e2c3; the focused regression test and pi-tui build pass. A PR will be opened after the required maintainer lgtm.

@greptile-apps

greptile-apps Bot commented Jul 31, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds inline image support to Orca's terminal panes by integrating the official @xterm/addon-image package, enabling Kitty graphics protocol and iTerm2 inline image (IIP) sequences. All three PTY spawn paths (daemon subprocess, local provider, relay handler) are updated to export ORCA_IMAGE_PROTOCOL=kitty, and the renderer is updated to create, load, and dispose an ImageAddon per pane with explicit memory limits.

  • Addon lifecycle: ImageAddon is instantiated in createPaneDOM with configured safety bounds (8 MB per payload, 4 MP pixel limit, 16 MB storage per pane with FIFO eviction), loaded in openTerminal, and disposed in disposePane following the same try/catch pattern used by all other addons.
  • Environment advertisement: ORCA_IMAGE_PROTOCOL: 'kitty' is injected into the PTY environment in pty-subprocess.ts, local-pty-provider.ts, and pty-handler.ts, allowing image-capable TUIs to detect Kitty graphics support without impersonating KITTY_WINDOW_ID or another terminal.
  • Tests: New focused tests verify the addon options, load order, and disposal on pane close across pane-dom-creation.test.ts and pane-lifecycle.test.ts, plus env-var assertions in all three PTY provider test files.

Confidence Score: 5/5

Safe to merge. Changes are renderer-only and additive: a new xterm addon is instantiated per pane with explicit memory bounds and disposed on pane close, and an environment variable is injected into all three PTY spawn paths to advertise capability to TUIs.

All three PTY spawn paths are consistently updated. The ImageAddon follows the same creation/load/dispose lifecycle pattern as every other addon in the codebase. Memory is bounded at multiple layers (per-image payload cap, pixel limit, per-pane FIFO storage cap). SIXEL is explicitly disabled. Test coverage verifies the constructor options, load order, and disposal. No IPC, authentication, filesystem, or network code was touched.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
src/renderer/src/lib/pane-manager/pane-dom-creation.ts Adds ImageAddon instantiation with bounded options (8 MB payload cap, 4 MP pixels, 16 MB storage, FIFO eviction). SIXEL is explicitly disabled. The addon is always created and assigned to the pane object.
src/renderer/src/lib/pane-manager/pane-lifecycle.ts Loads imageAddon after terminal.open() and before WebGL attachment (consistent with other addon ordering). Disposes it in disposePane() using the same try/catch pattern as all other addons.
src/renderer/src/lib/pane-manager/pane-manager-types.ts Adds optional imageAddon?: ImageAddon to ManagedPaneInternal. Not surfaced on the public ManagedPane type, consistent with webglAddon and ligaturesAddon.
src/main/providers/local-pty-provider.ts Adds ORCA_IMAGE_PROTOCOL: 'kitty' to the PTY spawn environment. Consistent with the other two spawn paths.
src/main/daemon/pty-subprocess.ts Adds ORCA_IMAGE_PROTOCOL: 'kitty' to the daemon subprocess environment, advertising Kitty graphics capability to TUIs running under the daemon path.
src/relay/pty-handler.ts Adds ORCA_IMAGE_PROTOCOL: 'kitty' to the relay PTY spawn environment, completing the three-path coverage for the environment variable advertisement.
src/renderer/src/lib/pane-manager/pane-lifecycle.test.ts New test verifies loadAddon:image is called during openTerminal and that imageAddon.dispose() is called exactly once on pane disposal.
src/renderer/src/lib/pane-manager/pane-dom-creation.test.ts New test verifies that all ImageAddon constructor options match the intended bounds (pixelLimit, storageLimit, size caps, sixelSupport: false, iipSupport: true, kittySupport: true).

Sequence Diagram

sequenceDiagram
    participant Shell as Shell / TUI
    participant PTY as PTY Provider
    participant Renderer as Renderer createPaneDOM
    participant xterm as xterm.js Terminal
    participant ImageAddon as ImageAddon

    PTY->>Shell: "spawn(env: {ORCA_IMAGE_PROTOCOL: 'kitty', ...})"
    Note over Shell: TUI detects Kitty graphics support

    Renderer->>ImageAddon: "new ImageAddon({pixelLimit, storageLimit, kittySizeLimit, iipSizeLimit, ...})"
    Renderer->>xterm: new Terminal(opts)

    Renderer->>xterm: terminal.open(xtermContainer)
    Renderer->>xterm: loadAddon(fitAddon)
    Renderer->>xterm: loadAddon(imageAddon)
    Renderer->>xterm: loadAddon(searchAddon, ...)
    Renderer->>xterm: attachWebgl(pane)

    Shell-->>xterm: Kitty / iTerm2 image escape sequences
    xterm->>ImageAddon: decode and render (RGBA canvas overlay)
    Note over ImageAddon: Bounded: 8MB payload, 4MP pixels, 16MB/pane FIFO

    Note over Renderer: On pane close
    Renderer->>ImageAddon: imageAddon.dispose()
    Renderer->>xterm: terminal.dispose()
Loading

Reviews (2): Last reviewed commit: "fix(terminal): advertise inline image ca..." | Re-trigger Greptile

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Adds @xterm/addon-image and stores a configured image addon on each managed pane. The addon enables bounded Kitty and iTerm2 image support, placeholders, and configured pixel and storage limits while disabling Sixel support. openTerminal loads the addon when present. disposePane disposes the addon and ignores disposal errors. Tests cover addon configuration, loading, and disposal.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Linked Issues check ⚠️ Warning The renderer support is implemented, but Pi Code and Kimi Code still reject Orca, so the linked issue is not complete end to end. Update the Pi Code and Kimi Code capability allowlists to recognize Orca as supporting inline images, or link verified follow-up changes that provide this behavior.
✅ Passed checks (4 passed)
Check name Status Explanation
Out of Scope Changes check ✅ Passed The dependency, addon configuration, lifecycle handling, and regression tests directly support inline image rendering and pane resource management.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check ✅ Passed The title clearly and concisely describes the main change: adding inline image support to the terminal.
Description check ✅ Passed The description covers the change, testing, review, security, platform scope, risks, and follow-up work in the required sections.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@AmethystLiang

Copy link
Copy Markdown
Contributor

thanks for the PR. will review it. since it touches terminal, it will have a longer review time as we ensure it doesn't cause any functionality or performance regression for the terminal

@XRX193

XRX193 commented Jul 31, 2026

Copy link
Copy Markdown
Author

@coderabbitai review

The linked-issue compatibility warning now has verified client follow-ups: MoonshotAI/kimi-code#2454 is open with 716/716 pi-tui tests passing, and earendil-works/pi#7357 links the tested Pi implementation while its required contributor approval is pending. Both preserve TERM_PROGRAM=Orca and enable Kitty image output client-side.

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

@XRX193 I will review the current changes, including the terminal image addon lifecycle and the stated client compatibility follow-ups.

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@AmethystLiang
AmethystLiang requested a review from OrcaWin July 31, 2026 16:49
@AmethystLiang AmethystLiang added the P2 Normal priority: nice-to-have or lower urgency label Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority: nice-to-have or lower urgency

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Inline image display

3 participants