Repository navigation
✨ feat(rendering): smooth annotations and refine mask outlines - #132
Conversation
Smooth overlays and label backgrounds while keeping media and text pixels intact. Mask outlines keep their screen width and heatmap focus follows visible anomaly coverage.
Offer Off, FXAA and FXAA + 2x in one persistent quality selector. Expose heatmap colours, threshold and opacity beside their rendered layer.
Document the annotation smoothing control, sharp labels, screen-space mask outlines and heatmap focus behavior.
Check unchanged media and glyph pixels, smoothed label chips, aligned focus coverage, screen-space borders and resource disposal in the owned headed browser. Retain the playback and human-scrub cost instruments.
Mega ReviewStatus: Eight Warnings block. Review-clean does not replace tests, typecheck or CI. Blocking
Non-blockingFour Minors
CaveatsStatic review of the exact pushed head. No builds or tests were run by this review. All branch fixes and features remain in scope. Ticket compliance was not requested. Review LedgerOpen findings
Prior key statuses None. Unresolved prior open findings None. Deliberate tradeoffs None. Refuted
Implausible None. Pre-existing
|
Masks stay visible while zoom updates their screen-pixel borders. Keep the current same-frame artifact prepared until its stroked RGBA replacement is ready, and reuse fills and ID rasters across display-width changes. Preserve fine raster quality during replacement and count retained stroke alignment storage toward the cache byte limit.
Mega ReviewStatus: All twelve prior findings are resolved. No Criticals or Warnings. Two Minors will be addressed before finishing the convergence loop. Fixed since last round
Non-blocking
CaveatsThis round reviewed the repair delta and all carried keys. Untouched parts were reviewed at Review LedgerOpen findings
Prior key statuses
Unresolved prior open findings None. Deliberate tradeoffs None. Refuted
Implausible None. Pre-existing
|
Mega ReviewStatus: No open findings. Both prior Minors are resolved; earlier resolved keys remain closed. Fixed since last round
CaveatsThis round reviewed the 225-line repair delta in three files plus both carried keys. The untouched PR was reviewed in earlier rounds, so the clean verdict covers the accumulated rounds rather than a fresh full pass. Two Codex search lanes, one Claude lane and a bounded carried-key verifier completed static inspection; all changed files were read in full. Historical comparisons reused byte-identical current spans. The review executed no tests or builds. Fresh real-browser pixel checks remain pending replacement of the closed reserved Chrome window. Ticket compliance was not requested. Review LedgerOpen findings None. Prior key statuses
Unresolved prior open findings None. Deliberate tradeoffs None. Refuted
Implausible None. Pre-existing
|
Changing mask styles after a fast scrub could leave the current paused frame at preview resolution indefinitely. Rearm its existing one-shot refinement when the style invalidates that artifact, while preserving inexpensive motion previews and disabled or disposed inactivity. Extend the existing tier regression test with repeated paused restyles, frame identity, and idle work checks. Verified the proof fails with the refinement rearm removed; all 144 affected tests, web typechecking, ESLint, and formatting pass.
Mega ReviewStatus: No open PR findings. All fourteen previous keys remain closed, and this repair introduces no Criticals or Warnings. Fixed since last round
Non-blocking
CaveatsThis round reviewed the 48-line repair delta in two files, their complete current contents and dependent call chains. Untouched PR surfaces retain the earlier rounds' verdicts, so the clean verdict covers the accumulated rounds. One fresh Codex lane, one Claude lane and the orchestrator completed the static review across all thirteen lenses. No tests, builds, browsers or performance benchmarks were run by this review. Ticket compliance was not requested. Review LedgerOpen findings None. Prior key statuses
Unresolved prior open findings None. Deliberate tradeoffs None. Refuted
Implausible None. Pre-existing
|
Keep annotation smoothing, sharp labels, consistent default outlines and paused mask refinement alongside current viewport, editing and still-image rendering behavior. Retain explicit mask and screen outline units with alignment-aware contours, and reuse bounded polygon rasters when preparing those contours. Validation: npm run verify passes, including 270 files and 3,560 tests. Compare the merged tree against both parents and retain all PR-only source files.
Keep Outside and Inside outlines on the correct side of mask holes when using explicit outline units. Pixi aligns each closed path to its own enclosed area, so inner contours need the opposite alignment. Use at most two stroke groups per detection and retain the centered batch. The default raster-shader path is unchanged. Validation: real Pixi line geometry for all three alignments; removing the hole reversal fails the Outside and Inside assertions. npm run verify passes 270 files and 3,563 tests.
Zooming an RGBA fallback mask with vector outlines was scheduling an unnecessary raster rebuild. Only borders baked into the raster depend on its display width, so exclude explicit screen/mask-unit outlines from that invalidation flag. Extend the existing zoom tests for both vector outline units, covering cached frames and preparation that completes after a zoom change. The four new cases fail before the fix and pass after it; baked-border replacement remains covered by the existing cases.
Keep annotation smoothing compatible with current multi-selection and media display controls. Include upstream selection preservation, gamma, still-image auto contrast, and the 0.2.8 release update. Retain both annotation and media-filter cleanup. The shared scene test supports their uniform groups while preserving texture resources.
2 participants
The top views locate the foreground horse; the pink box marks its ear edge. In the comparison rows, read left to right: main, this PR with AA off, then standard FXAA. All three use the same Horse frame, a 2px border setting, DPR 1, and settled 758px masks. The bottom row enlarges the same ear edge 8x. This is image zoom; rendering density stays at 1x.
The middle column already uses this PR's corrected border coverage. Compare middle to right for the extra FXAA effect; compare left to middle for the border repair. The video and label text stay outside the FXAA pass.
Original lossless renderer PNGs: main · PR, AA off · PR, standard FXAA.
Description
Add optional annotation smoothing and fix inconsistent mask border thickness, focus dimming over annotation outlines, rectangular anomaly focus cutouts, and square corners on standalone mask labels. Paused mask style changes also recover settled detail after fast scrubbing.
Optional annotation smoothing
Enable standard FXAA across annotation layers, including focus and interaction, with
annotationAntialiasing: true. Video pixels stay outside the smoothing pass; label backgrounds are smoothed while glyphs render at output density. Smoothing defaults to off.The public option is
annotationAntialiasing: false | true | 2. See presentation styles. All smoothing screenshots here use standard FXAA, without the optional higher density.Mask border width and placement
Interpret
stroke.widthas the total width in CSS pixels, across clip sizes, DPR changes, resize and camera zoom. On main, the same width setting looks thinner on the narrow Horse canvas than on the wide basketball canvas.The pink boxes locate the foreground horse's left mask edge and the center white player's torso. In both detail rows, left is main and right is this PR, with total 2 CSS px and the new Outside default. Both detail rows use the same 8x image zoom, so their border thickness can be compared. The overview sizes differ. This also changes the old border placement.
Support Outside, Center, and Inside placement, with Outside as the default. See mask styles.
The pink box locates the left edge of the foreground horse. In the detail row, all three outlines have the same total 2px width. Outside adds cyan over the background; Inside replaces blue mask fill; Center splits the width across the boundary. These are the same pixel coordinates at 8x image zoom.
AA off bypasses FXAA; the mask border still uses fractional pixel coverage. A straight stroke aligned to the output pixel grid can have crisp, fully opaque border pixels. When an edge falls between pixels or runs diagonally, partly covered pixels blend with the fill or background. A 2px stroke may therefore cover two full pixels, or a shifted stroke may have coverage like ½ + 1 + ½. Its total width remains 2 CSS px. There is no separate hard-edged mask mode in this PR; mask IDs still use nearest sampling.
Focus below annotation outlines
Draw focus dimming below mask and box outlines so their borders retain color and contrast. Keep focus targets tied to the current detection frame.
The pink box marks the top-right corner of the foreground horse's bounding box. Inspect that corner in the 8x detail row. Main darkens its border along with the background; this PR preserves its brightness. Both sides use the same 40% focus dimming, with FXAA off. The adjacent mask edge also stays above the dimming layer.
Heatmap focus follows visible anomalies
Use visible anomaly pixels, including the configured cutoff and transparency, for the focus opening. Empty heatmaps leave no rectangular opening. Disable rectangle fallback with
BaseFocusStyle({ shape: null }); the Pebbles demo uses this setting. See heatmap styles.The whole-frame overview locates the bolt on the pebbles. In the four detail panels, top left: the unwanted bright rectangle extends beyond the bolt. Top right: this PR limits the opening to visible heat. Bottom left: standard FXAA smooths that same opening. Bottom right: opacity 0 removes heat and leaves no opening. The source video and heatmap are already soft; this 3x image zoom exposes their existing pixels.
Label corners follow the visible box
Round all corners when no visible box touches the label, including standalone mask labels. Square only attached corners: bottom corners above a box, top corners below it. Preserve explicit corner-style overrides. See label styles.
The overview locates the horse's standalone label; the cow inset shows a label below its box. In the first detail row, the standalone horse label gains rounded bottom corners. In the second detail row, only corners that meet a visible box stay square. The label above a box has square bottom corners; the cow label below a box has square top corners. All four crops use 5x image zoom and FXAA off.
Bounded polygon preparation
Rasterize polygons directly at the bounded target resolution, avoiding full-resolution mask intermediates. Keep prepared polygon outlines consistent with display density and viewport scale.
The pink box locates the foreground yellow player's legs. Compare the steps in the yellow leg outline with FXAA off, then their blending with standard FXAA. Masks are disabled. The bottom row uses 8x image zoom. This shows rendered appearance; it does not measure preparation speed.
Paused mask style changes recover settled detail
After fast scrubbing, changing a mask's fill or outline while paused can leave the current frame at preview resolution. Rearm its existing 150 ms refinement when the style invalidates the prepared mask. The current frame returns to settled detail, while motion previews and idle behavior remain inexpensive.
The pink box marks the horse ear used in the 8x detail row. Both pages had settled 758px masks before the same paused fill change, from 55% to 65%. Main stays at a 380px preview; this PR returns to 758px detail. Inspect the larger steps on the left. FXAA is off. This is the only mask comparison intentionally showing preview versus settled detail.
Demo controls
These are actual panel captures at their original pixel size. Quality controls global FXAA; Segmentation controls mask detail and border placement; Heatmap controls anomaly appearance; Focus controls rectangle fallback.
Type of Change
Validation
npm run verifyoned0b5e9: 3,489 tests passed, 10 skipped; lint, types, builds and package checks passed0b5e9; no open PR findingsNotes For Reviewers
Before/after screenshots compare main code at
466a2c7with this PR ated0b5e9, in headed Chrome using WebGPU. Within each comparison, frame, viewport and relevant settings match. Page DPR and renderer DPR are both 1. Mask examples use settled fitted rasters, except the explicitly labeled main paused-restyle failure. All smoothing examples use standard FXAA.Every comparison is a lossless 640px-wide PNG. Renderer images come directly from the canvas before browser compositing. Reduced whole-frame overviews and pink crop markers locate the subject; actual-size insets retain the captured pixels. Detail comparisons use labeled integer nearest-neighbor magnification, so image zoom does not add blur or change rendering density. Panel captures are unresized. These screenshots show visual behavior, not performance measurements.
Mask borders may move outward because width now means total CSS-pixel thickness and the default alignment is Outside. Fractional pixel coverage also changes AA-off edges. Outline work is bounded to 16 mask texels per side, which can limit thickness at very small fitted sizes.
Smoothing is opt-in. The optional
annotationAntialiasing: 2increases preparation work, cache bytes, and GPU memory use; capture sizes stay within the renderer's texture limit. Real-renderer checks live in tools/annotation-aa.