Skip to content

core: lay out an undrawn pane at its display box - #142

Open
jsegaran wants to merge 1 commit into
yas-run:mainfrom
indent-com:core/undrawn-pane-display-box
Open

jsegaran wants to merge 1 commit into
yas-run:mainfrom
indent-com:core/undrawn-pane-display-box

Conversation

@jsegaran

@jsegaran jsegaran commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

What was observed

A resizable YasSurfaceCanvas that remounts, re-attaches or reconnects has a display size before it has drawn a frame. During that gap the first click often doesn't reach the remote app. We saw this in a browser-panel fuzz harness running against a real yas 0.5.3 server and Chrome. Instrumenting it showed the press does reach handleMouse. drawnGeometry() returns a geometry and the press goes on the wire, but that geometry comes from the wrong box:

  • attach() allocates a 640×480 placeholder backing buffer.
  • layoutCanvasBox treats that placeholder as a frame and sizes the CSS box to 640×480 (divided by DPR).
  • A press outside that box never reaches the canvas.
  • A press inside it is sent as a fraction of the placeholder, not the pane. In an 800×600 pane, a press at (444,333) went out as 0.694 instead of 0.555.

Root cause

Before its first frame, a display-sized view lays out and maps input against attach()'s 640×480 placeholder instead of its display box.

Fix

While no frame has been drawn (_lastFrameSize is null), layoutCanvasBox fills the display box with object-fit: fill. drawnGeometry() already reads that box and its fill mode, so a press anywhere in the pane reaches the canvas and maps against the pane. A press at the pane centre goes out at 0.5,0.5. When the first frame lands, presentFromStore records _lastFrameSize and re-runs layout, so the existing frame-based geometry takes over. clearPresentation() resets _lastFrameSize, so a reused pane gets the same treatment.

Unchanged:

  • views without a display size, including thumbnails and the provisional pre-size layout
  • frame-based layout after the first frame

The drawnGeometry guard

handleMouse returns before any focus or modifier side effect when drawnGeometry() is null. It also freezes the geometry used for both halves of a click. This protects a zero-size or remounting canvas from emitting half a click, and stops a later layout change from moving the press. The guard is untouched. A zero-size box still yields no geometry, and "does not focus or press through a zero-size canvas" still passes. The down and the up of a pre-frame press use the same display-box geometry.

Tests

  • New repro: maps a press before the first frame against the display box. A display-sized 1200×700 view with no drawn frame must lay out its canvas at 1200×700 and send a centre press as down and up at 0.5,0.5. On main it fails with Expected "1200px", Received "640px".
  • Existing tests: the layout and pointer tests faked a frame by writing canvas.width/height directly. That is indistinguishable from the placeholder, so they now use a small drawFrame helper that also records _lastFrameSize, as presentFromStore does. Two tests encoded the old placeholder behaviour and were updated:
    • keeps a smaller frame 1:1 at the top-left now draws an explicit 640×480 frame.
    • positions a sized view absolutely… now expects the 800px display box.

Commands run:

  • cd js && pnpm --filter @yas-run/core exec vitest run src/__tests__/YasSurfaceCanvas.test.ts: 270 passed. With only the test changes applied, the new repro fails.
  • cd js && pnpm --filter @yas-run/core run test: 86 files, 1537 passed, 22 skipped.
  • cd js && pnpm --filter @yas-run/core run typecheck: clean.
  • ./bin/fmt --check: clean.

End-to-end, I ran the fuzz harness with a scripted remount/reattach/drop sequence (12 steps) on a 1300×800 pane against a yas 0.5.3 server and Chrome, then repeated it with this branch's built @yas-run/core. js/core differs from v0.5.3 only in yas/fs.

Step Before After
remount 4/4 first clicks lost 0/4 lost
reattach 0/4 lost 0/4 lost
drop 4/4 lost 4/4 lost (see below)

Known follow-ups (not in this PR)

  • After a dropped link, the press now goes out at the right coordinates on a connection the client hasn't yet noticed is dead. The client logs "connection lost, re-attaching" right afterwards. That's a transport-liveness issue.
  • YasNativeWorkspaceConnection.sendSurfacePointer silently drops input, both down and up, when the surface view isn't open yet.

View in Indent
Tag @indent to continue the conversation here.

Until a frame is drawn, a display-sized view's canvas still holds attach()'s
640x480 placeholder, and layoutCanvasBox sized the CSS box from it. A press
outside that box missed the canvas, and a press inside it was sent as a
fraction of the placeholder rather than of the pane. Fill the display box
until the first frame lands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant