feat: desktop-pet click-through with a per-platform mode split - #61
Open
AUDOSt0ck1ng wants to merge 3 commits into
Open
feat: desktop-pet click-through with a per-platform mode split#61AUDOSt0ck1ng wants to merge 3 commits into
AUDOSt0ck1ng wants to merge 3 commits into
Conversation
The transparent, always-on-top avatar window can now pass clicks through to whatever sits behind it, from a tray toggle that starts off on every platform so nothing changes for anyone who does not ask for it. `setIgnoreMouseEvents` carries no platform restriction, but its `forward` option is macOS and Windows only, and forwarding is the only reason an ignoring window still receives mouse moves. That splits the feature in two rather than into supported and unsupported: those two platforms carve the character back out of an ignoring window, while elsewhere the whole window passes through until the toggle comes off. Where the character is gets decided by sampling one pixel of alpha from the frame just drawn, inside `scene.onAfterRender` so it needs no `preserveDrawingBuffer`. A raycast would bone-transform every vertex of a ~29k triangle skinned rig on the CPU per test, well past a frame's budget; sampling costs the same whatever the model, and alpha-cut hair passes clicks through the gaps it appears to have. An active gesture keeps the window whatever lies under the cursor, so that answer short-circuits ahead of the synchronous read rather than stalling the pipeline once per frame of a drag. The main process owns the mode and treats the renderer as advisory: results in the wrong mode, while the mode is off, or of any type but a boolean never reach the window. Changing the flags drops the window out of the always-on-top band and disturbs its workspace visibility, so both are re-asserted with every change. Design and original implementation from xikhar#45. Co-authored-by: zyx_281795 <129176228+zyx281795@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Click-through was session state that every launch reset. It now persists as `click_through_enabled` in the settings snapshot, and Settings → Appearance carries the same toggle the tray menu has, as asked for in xikhar#57. The mode beside it stays out of the store on purpose: it follows the running platform, so storing it would let a copied settings file claim a silhouette that the machine reading it cannot deliver. Settings reads it through `persona:settings-get-click-through-mode` instead, leaving `clickThroughModeFor` the only place the rule lives. Both entry points run through the same `setClickThroughEnabled`, which applies the window flags, tells the renderer, refreshes the tray, and writes the store together, so neither can leave the other stale. The stored choice is seeded before the avatar window is created, so its first flags already match rather than flipping once something notices. The tray callback catches a failed write, since a menu handler has nowhere to reject to and the flags have already changed by then; the Settings path keeps rejecting, and its notice reports what happened. `clickThroughCopy` keeps the per-mode wording out of the component and under test, including the caveat that Linux is X11-verified only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The hit-test only ran on a frame a pointer event had asked for, but the character keeps moving under a cursor that is standing still. Park the cursor in a gap beside the character, let the idle sway bring the character under it, and the click still passes through to the desktop; park it on the character, let the character sway away, and the avatar window swallows a click meant for the app behind it. Either way the answer came from a frame the animation had already walked away from. Sampling every presented frame is affordable. Measured on Windows at 60Hz across two configurations differing in window size and lighting, the read waits 1.6-4.5ms on the GPU while frame times stay on 16.7ms with no long frames at all: the wait replaces the one the frame would otherwise spend at vsync rather than adding to it. A throttle would buy nothing and cost staleness. A shorter frame budget is unmeasured, and is the first thing to check if the mode ever reads as janky. Two things the frame loop needed once it stopped waiting to be asked. It holds off until a forwarded move says where the cursor is, rather than answering every frame for the seeded origin, since whether that corner is transparent is a fact about the current framing. And a cancelled pointer now ends the gesture: a compositor takeover or a lifted touch contact leaves no `pointerup`, and for touch no further `pointermove`, so a gesture cleared only by those two would pin the window interactive with the tray toggle as the only way out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
After the change, the setting is persisted, and the function behaves exactly as it did before during testing on my computer. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #57.
Lets the transparent, always-on-top avatar window float over the desktop: while click-through is on, clicks around the character reach whatever sits behind it. Design and original implementation from #45 by @zyx281795; this takes the click-through half forward against the review it received, and follows the direction agreed in #57.
What is here
silhouette)whole-window)Off by default on every platform, so nothing changes for anyone who does not ask for it. The choice persists across launches and is exposed under Settings → Appearance alongside the tray toggle. Linux is marked experimental in the README, in Settings, and in
docs/DEVELOPMENT.md: it relies on the X11 input shape and no Wayland compositor has been verified.The platform constraint
setIgnoreMouseEventscarries no@platformannotation, butIgnoreMouseEventsOptions.forwardis@platform darwin,win32. Forwarding is the only reason an ignoring window still receives mouse moves, so this is not supported vs unsupported: every platform can pass clicks through, and only the selective form needs forwarding.clickThroughModeForis the one place that rule lives.How the hit test works
One pixel of alpha from the frame just drawn, sampled in
scene.onAfterRender, rather than a CPU raycast. The packaged rig is ~29k triangles across three skinned meshes and three.js bone-transforms every vertex per cast, which is well past a frame's budget; sampling is O(1) in model complexity and matches what the user sees, since alpha-cut hair then passes clicks through the gaps it appears to have.Every presented frame is sampled, not only the frames a pointer event asks for, because the character keeps moving under a cursor that is standing still. Measured on Windows at 60Hz across two configurations differing in window size and lighting, the read waits 1.6–4.5 ms on the GPU while frame times stay on 16.7 ms with no long frames: that wait replaces the one the frame would otherwise spend at vsync rather than adding to it. A display with a shorter frame budget is unmeasured, and is the first thing to check if the mode ever reads as janky.
Ownership
The main process owns the mode and treats the renderer as advisory: results arriving in
whole-windowmode, while the mode is off, or of any type but a boolean never reach the window. The mode reaches the renderer both ways — pushed on change so the tray toggle takes effect at once, and pulled throughpersona:get-click-throughon mount so a reload still learns it. It is deliberately never theget-snapshot"last event".The tray toggle and the Settings control run through one
setClickThroughEnabled, which applies the flags, tells the renderer, refreshes the tray, and writes the store together. Whether click-through is on persists asclick_through_enabled; the mode beside it does not, since it follows the running platform and a stored copy could claim a silhouette the machine reading it cannot deliver.Tests and docs
Node covers mode selection, flag derivation, hit-test gating, the bridge surface, and the settings round-trip including that a developer reset leaves this Appearance setting alone. Vitest covers drawing-buffer mapping, the alpha threshold, and the per-mode Settings copy. README,
docs/DEVELOPMENT.md, and thedocs/RELEASING.mdmanual checklist are updated.Still open
setIgnoreMouseEvents(true)under Wayland/Hyprland remains unverified — X11 should work through the input shape region, but Electron does not document Wayland and I have no way to test it. Confirmation from someone on that setup would help, and the experimental marking should stay until then.🤖 Generated with Claude Code