Performance: ~92% of CPU is QPainter software rasterization; sluggish pan drag is a full-window sibling repaint

[jensenpat]

Summary

CPU profiling of a live AetherSDR session shows that ~92% of process CPU is Qt software rasterization (QPainter/QRasterPaintEngine), not DSP and not the GPU spectrum path. Two scenarios were captured:

1. Idle / steady-state RX — cost concentrated in QPainter-rendered widgets (audio scope WaveformWidget, plus S-meter and VFO).
2. Panadapter drag (the user-visible "sluggish drag" symptom) — total CPU roughly doubles, and the trace exposes the root cause: each drag frame triggers a full-window backingstore repaint that re-rasterizes sibling widgets (scope, meters, and QPushButtons) and re-shapes all text from scratch via HarfBuzz every frame.

The GPU spectrum/waterfall (QRhiMetal) and the audio DSP pipeline are both cheap and healthy. This issue documents the full profiler report and proposes optimizations across widget, GPU, panadapter, and audio subsystems.

Profiling/analysis only — no code changed. Refresh-rate defaults and software-vs-GPU rendering are UX/architecture decisions, so this is filed for maintainer review per the autonomy boundaries in CLAUDE.md.

---

Methodology

• Tool: xcrun xctrace record --template "Time Profiler" --attach <pid> (Instruments' Time Profiler engine, CLI), attached to a live session.
• Captures: 25 s idle/steady-state, and 120 s while actively dragging the panadapter.
• Symbolication: against the local RelWithDebInfo build.
• Aggregation: exported the time-profile table to XML; aggregated weight per thread, per thread-state (running vs. waiting), per leaf symbol, and inclusive per paint/event entry frame.
• Environment: Apple Silicon (SoC t8142), macOS 26.5 (arm64), Qt 6.11.0 (Homebrew).
• Note: absolute ms = relative weight, not a worst-case ceiling. Idle window averaged ~0.97 cores; drag window averaged ~1.49 cores.

---

Per-thread CPU breakdown

All active threads sampled at 100% "Running" — genuinely on-CPU, no lock contention / priority inversion observed. The bottleneck is main-thread throughput, not blocking.

Idle (25 s window)

Thread	On-CPU	Share	Role
Main thread	9,047 ms	37.5%	QPainter software rasterization
5× Thread (pooled)	13,346 ms	55.3% combined	Qt raster engine parallel span fills
AudioEngine	364 ms	1.5%	Opus + r8b resampler + EQ / NR DSP
PanadapterStream	287 ms	1.2%	VITA-49 parse + signal emit
AetherSDR (Metal/GCD ×4)	~1,060 ms	~4%	QRhiMetal GPU submit + libdispatch

Panadapter drag (120 s window) — total CPU ~doubles

Thread	On-CPU	Share	Δ vs idle
5× Thread (pooled)	85,879 ms	48.1%	pinned ~0.72 cores
Main thread	77,160 ms	43.3%	362 → 643 ms/s
AudioEngine	3,231 ms	1.8%	unchanged rate
PanadapterStream	3,075 ms	1.7%	unchanged rate
com.apple.NSEventThread	855 ms	0.5%	input delivery is not the bottleneck

The 5 pooled threads are not DSP workers

They are Qt's raster paint engine auto-threading large fills (QThreadPool): blend_untransformed_argb, comp_func_solid_SourceOver_neon, blend_color_argb. Critically, the main-thread raster engine dispatches spans to the pool and then blocks waiting for them, so main + pool is effectively one synchronous workload (~93% of CPU), and every paint stalls the event loop for the full rasterize-and-fill duration.

---

Root cause of the sluggish panadapter drag

The drag trace shows the full main-thread repaint chain (inclusive):

QApplication::notify                         13,900 ms
└─ QMainWindow::event                        11,426 ms
   └─ QWidgetRepaintManager::sync/flush      11,357 ms   ← full backingstore repaint cycle
      └─ QWidgetPrivate::drawWidget           9,434 ms
         └─ paintSiblingsRecursive            6,464 ms   ← THE smoking gun
            ├─ WaveformWidget::paintEvent     2,858 ms   ← audio scope (software QPainter)
            ├─ SpectrumWidget renderGpuFrame  2,376 ms   ← expected (pan moves)
            ├─ QPushButton::paintEvent        1,927 ms   ← buttons repainting during a pan drag (!)
            └─ SMeterWidget::paintEvent         750 ms

paintSiblingsRecursive at 6.5 s is the key finding. Each drag frame does not merely redraw the panadapter — Qt walks and re-rasterizes the widgets adjacent to it in the invalidated region: the audio scope, the meters, and a panel of QPushButtons. This is the classic signature of either:

• non-opaque widgets (missing Qt::WA_OpaquePaintEvent / autoFillBackground) forcing Qt to repaint siblings behind/around the dirty region, or
• a frequency-change signal cascade calling update() over a region wide enough to overlap those siblings (doActivate signal emission = 1,721 ms during the drag).

So the scope/meters are real contributors but collateral — pulled in by an over-broad repaint, alongside buttons that should never redraw during a pan drag.

Second drag-specific cost: text re-shaped every frame

Largely invisible at idle, significant during drag:

QTextEngine::shapeTextWithHarfbuzzNG          779 ms (main)
QPainterPrivate::drawTextItem                 758 ms
QRasterPaintEngine::drawCachedGlyphs          673 ms
QCoreTextFontEngine glyphIndex/loadAdvances/stringToCMap  ~520 ms

Text is HarfBuzz-shaped from scratch on every repaint — not just the VFO frequency (which legitimately changes during a drag) but apparently static button/meter labels too. Shaping is one of the most expensive per-frame operations and is almost entirely cacheable.

---

Where the raster cost originates (idle inclusive attribution)

Paint entry (your code unless noted)	Inclusive
QWidgetPrivate::drawWidget (compositing root)	2,222 ms
QRasterPaintEngine::fill	1,012 ms
WaveformWidget::paintEvent	991 ms
WaveformWidget::drawGraph	836 ms
QRasterPaintEnginePrivate::rasterize	825 ms
SMeterWidget::paintEvent	249 ms
QRasterPaintEngine::drawTextItem	142 ms
VfoWidget::paintEvent	120 ms
SpectrumWidget::renderGpuFrame (GPU path)	89 ms ✅

Hottest leaf symbols are all antialiased polygon rasterization (gray_set_cell 411 ms, gray_render_scanline/line, qt_alphamapblit, qt_memfill32).

---

Subsystem analysis & proposed optimizations

1. Repaint region / compositing (NEW — highest leverage for drag latency)

The single biggest lever for the sluggish drag is to stop the drag from invalidating sibling widgets. Fixing this collapses paintSiblingsRecursive (6.5 s) plus the button/scope/meter repaints (~5.5 s) that ride on it.

• Audit what the panadapter drag invalidates — ensure mouse-move handling updates only the spectrum/overlay region, not a region (or a parent) overlapping the scope, meters, and button panel.
• Ensure neighboring widgets set Qt::WA_OpaquePaintEvent (and/or opaque autoFillBackground) so Qt does not repaint them as siblings of the dirty region.
• Check the frequency-change signal path (doActivate 1.7 s during drag) — coalesce/limit the widgets that update() per mouse-move so a single drag step doesn't fan out into a full-window repaint.
• Confirm QPushButtons are opaque and not in the invalidated region — they should never repaint during a pan drag.

2. Widgets (highest leverage for overall CPU)

WaveformWidget::drawGraph (src/gui/WaveformWidget.cpp:447) per frame builds four full-width QPainterPaths (peak/RMS top/bottom, one node per pixel column), issues a per-column drawLine, draws them antialiased, and (in drawEnvelope) does a full-plot alpha fillPath — at up to 24 Hz (WaveformWidget.h:113).

• Batch per-column min/max bars into one drawLines(QVector<QLineF>).
• Replace the four QPainterPaths with drawPolyline() over prebuilt QPointF arrays (materially cheaper to rasterize).
• Disable AA on the dense vertical min/max bars; keep AA only on the thin peak/RMS traces.
• In drawEnvelope, replace the full-plot alpha fillPath with a precomputed gradient brush, or make it optional.
• Lower / make adaptive the 24 Hz default (e.g. 15 Hz, or back off on near-silent audio).
• Structural win: port WaveformWidget to the existing QRhi/GPU path (infra already exists for SpectrumWidget); a scope is a trivial GPU workload and would erase most of the combined ~22 s raster cost.

SMeterWidget / VfoWidget:

• Cache shaped text (QStaticText / cached glyph runs); re-shape only when the value string changes. ~2 s recoverable during drag.
• Verify repaint cadence vs. MeterSmoother output rate.

3. GPU (SpectrumWidget / QRhiMetal) — healthy, minor tuning

GPU path is efficient (renderGpuFrame 89 ms idle; ~2.4 s during a 120 s drag, expected since the pan moves).

• Skip beginPass / frame submission when there is no new FFT/waterfall data (avoid a render pass per vsync when the model is unchanged).
• Confirm waterfall uses a ring/scroll texture rather than full-texture re-upload per row (enqueueSubresUpload ~194 ms during drag).
• If the scope moves to GPU (§2), validate one swapchain present per vsync and no Metal queue oversubscription.

4. Panadapter / VITA-49 (PanadapterStream) — healthy, watch allocations

Low weight (287 ms idle, 3.1 s during drag), dominated by expected processDatagram, recvmsg/recvfrom, and queued-signal dispatch.

• Reuse a pooled buffer for the per-row QList<float> instead of allocating per datagram (QList<float>::fill + per-datagram QQueuedMetaCallEvent show up) — matters more at multi-pan / fast-waterfall rates.
• Re-profile under multiple panadapters + fast waterfall.

5. Audio (AudioEngine) — healthy

Only 364 ms idle / 3.2 s during drag (rate unchanged), spread across expected DSP: op_pvq_search_c/celt_pitch_xcorr_c (Opus), r8b::ooura_fft/CDSPBlockConvolver (resampler), ClientEq::process, tanhf (tube/saturation), remove_doubling. CoreAudio IO thread never blocks.

• Optional: replace per-sample tanhf in the tube stage with a polynomial/LUT if TX CPU ever becomes a concern.
• Re-profile during TX (Opus encode + tube pre-amp + RN2/NR2/NR4/DFNR).

---

Suggested priority

1. Narrow the drag repaint region (§1) — biggest win for the user-visible sluggish drag; collapses ~12 s of sibling repaints + button redraws.
2. WaveformWidget paint cost (§2) — batched drawLines + polyline + selective AA (safe, no design change), then evaluate GPU port.
3. Cache shaped text in VFO + S-meter (~2 s during drag, easy).
4. Refresh-rate review for scope + meters (UX decision).
5. GPU skip-frame-when-unchanged + panadapter buffer pooling (forward-looking).

Follow-ups / additional profiling

• TX path (Opus encode + tube pre-amp + RN2/NR2/NR4/DFNR).
• Multiple panadapters + fast waterfall.
• Idle/near-silent audio (does the scope still repaint at 24 Hz?).
• Allocations (Instruments "Allocations") to quantify per-frame malloc churn in paint + VITA-49.

Both .trace captures are reproducible with the xctrace command above; happy to attach exported summaries or a focused sub-trace on request.

[aethersdr-agent]

Triage: maintainer-review

Thanks for the rigorous profile — methodology, attribution, and proposed fixes are all well-grounded. I verified the key code references against main:

• WaveformWidget::drawGraph is at src/gui/WaveformWidget.cpp:449 (the per-column drawLine loop, four QPainterPaths, and the setRenderHint(Antialiasing, true) toggle are all exactly as described — lines 463–502).
• drawEnvelope builds a closed fillPath and fillPaths it antialiased at lines 568–581.
• Default scope refresh is m_refreshRateHz{24} at src/gui/WaveformWidget.h:113.
• SMeterWidget::paintEvent and VfoWidget::paintEvent both enable AA and re-drawText numeric readouts every frame (no glyph-pixmap cache), consistent with the ~340 ms of glyph rasterization the trace attributes to them.

Root-cause summary

One workload — main thread + 5 Qt raster-pool threads (≈93% combined) — is CPU rasterizing antialiased QPainterPaths for WaveformWidget/SMeterWidget/VfoWidget. The GPU spectrum path (QRhiMetal, 89 ms inclusive) and the DSP/IO threads are healthy. The hottest leaves (gray_set_cell, blend_color_argb, qt_alphamapblit_argb32) all point at AA polygon scan-conversion, which is exactly the cost path QPainterPath + setRenderHint(Antialiasing) produces.

Recommended fix plan (in priority order)

I'd group this into three PRs of escalating scope so each is independently reviewable:

PR 1 — WaveformWidget paint micro-opts (safe, no UX change)

• src/gui/WaveformWidget.cpp:469-493 — replace per-column drawLine with a single drawLines(QVector<QLineF>); replace the four QPainterPath paths with drawPolyline() over pre-sized QPointF arrays (cached members to avoid per-frame alloc).
• Toggle AA off for the dense vertical min/max bars; keep AA only for the four peak/RMS polylines (lines 495–501).
• In drawEnvelope (lines 568–581) make the alpha fillPath optional, or pre-compute a vertical gradient brush and fillRect clipped by the rms polyline.
• Same drawLines batching for the clip ticks at lines 504–515 and 601–611.

PR 2 — Glyph caching in SMeterWidget and VfoWidget

• Cache the numeric readout strings as QPixmaps keyed by (text, font, color), re-rendered only when the string changes (the ~340 ms drawTextItem/drawCachedGlyphs/glyphIndex cost is unchanged-glyph re-rasterization). Confirm meter update() cadence matches MeterSmoother change rate — re-paints firing faster than the value changes are pure waste.

PR 3 — UX/architecture decisions for maintainer

• Lower the default m_refreshRateHz (24 → 15?), and/or back it off adaptively when the audio is near-silent. Needs a UX call.
• Port WaveformWidget to the existing QRhi path used by SpectrumWidget. Bigger change, larger upside (~22 s combined raster cost evaporates), but needs architecture sign-off because it adds a second GPU widget and changes the rendering model for the scope. The reporter is right that a scope is a trivial GPU workload (line strip + fill).

Items 4 (QRhiMetal::beginPass skip-when-unchanged, waterfall ring texture, PanadapterStream buffer pooling) are forward-looking and best deferred until multi-pan/TX profiles are captured — they're not on the current hot path.

Handoff questions for the maintainer

1. OK to authorize PR 1 + PR 2 (the safe micro-opts and glyph caching) under aetherclaude-eligible? These don't change behavior or defaults and should claw back a large fraction of the cost.
2. Do you want to drop the 24 Hz scope default in the same pass, or keep that as a separate UX decision?
3. Is a GPU port of WaveformWidget something to scope now, or revisit once PR 1's gains are measured?

Holding off on any code changes pending your call.

---

_{🤖 aethersdr-agent · cost: $3.1355 · model: claude-opus-4-7}

[jensenpat]

Update: controlled A/B + idle profiling — drag is a victim of continuous repaint saturation, not the cause

Captured three xctrace Time Profiler traces on the same machine/session (Apple silicon, 5K-class window), normalized to ms of on-CPU time per second:

• Baseline — clean origin/main, 90 s, continuous slice/pan drag
• Fix — a per-mouse-move emit-coalescing change (throttle incrementalTuneRequested/centerChangeRequested to ~60 Hz), 90 s, identical drag
• Idle — clean origin/main, 18 s, zero interaction (mouse untouched)

Results (ms/s)

Metric	Baseline (drag)	Fix (drag)	Idle (no touch)
Total on-CPU	1.48 cores	1.46 cores	1.69 cores
Main thread	661	692	682
Pooled (Qt threaded raster)	677	628	871
QWidgetRepaintManager::sync	110	117	183
paintSiblingsRecursive	60.6	63.1	78.8
WaveformWidget::paintEvent (scope)	33.2	35.7	74.6
QPushButton::paintEvent	18.2	19.9	31.8
shapeTextWithHarfbuzzNG	10.9	11.2	11.7
SpectrumWidget::renderGpuFrame (GPU)	19.7	19.1	5.7
drag emit (centerChange/incrementalTune)	0.2	0.1	—

Conclusions

1. The emit-coalescing fix does nothing (baseline vs fix is within run-to-run noise). The drag emits cost ~0.1 ms/s — throttling free work saves nothing, and macOS already pre-coalesces mouse-move events. This change has been reverted.

2. The app burns ~1.55–1.7 cores on continuous whole-window QPainter software rasterization at idle, with nothing happening. Idle is actually higher than drag. Drivers:

   • The audio scope (WaveformWidget) at 24 Hz — 74.6 ms/s idle, each repaint triggering a full backing-store sync.
   • The translucent VfoWidget acting as an amplifier: it can't be cached, so every sync whose region overlaps it re-runs QPushButton::paintEvent for all its buttons (31.8/s) and HarfBuzz-reshapes their text (11.7/s).
   • Qt fans the resulting fills across the 5-thread pool (871 ms/s).

3. The drag stutter is a symptom of main-thread saturation, not drag cost. During a drag the GPU pan (renderGpuFrame) rises 5.7 → 19.7 ms/s as expected, but the main thread is already saturated by the idle repaint loop, so the pan can't get cycles to present → stutter. On a 5K/large window it's worse only because each of those continuous repaints covers ~2–3× more device pixels.

4. Logging is not a factor — no logging thread appears among the top CPU consumers (it's correctly off the main thread).

Mechanism (not parent/child — shared backing store)

The scope and S-meter are not children of the VFO (SMeterWidget is in AppletPanel, WaveformWidget in WaveApplet). The cascade is at the top-level backing-store level: multiple independent animation timers (scope 24 Hz, meters, waterfall, VFO ESC meter) each issue their own update() → sync (hence 183/s, far above one-per-vsync). Any sync whose dirty-region union overlaps the translucent VfoWidget repaints its entire button cluster + text.

Revised fix directions (data-justified, in leverage order)

1. Reduce the translucent VfoWidget per-repaint cost — give the button cluster / panel an opaque cached backing, or cache button label glyphs (QStaticText), so a sync touching its region stops re-rasterizing + re-shaping everything. (Visual-design adjacent — needs maintainer sign-off.)
2. Coalesce the many per-timer update() calls to one backing-store flush per vsync (183/s → ~60/s).
3. Move the audio scope (WaveformWidget) to the GPU/QRhi path (infra already exists for SpectrumWidget); eliminates the largest single software-paint contributor and its sync storm.
4. The §2 scope QPainter micro-opts (batched drawLines, polyline, selective AA) still compound.

Next: pinpoint the exact dirty-region root that produces the full-window paintAndFlush, and prototype (1)/(2).

[jensenpat]

Lean mode experiment — full results, and a correction on what actually moves the needle

Built a reversible, persisted "Lean" render mode (button in each panadapter's overlay menu) bundling every CPU driver we'd identified, then profiled it exhaustively with xctrace. Headline: Lean is a reproducible main-thread win (responsiveness), but it does not reliably reduce total CPU — the dominant pooled cost is a structural QRhiWidget compositing tax dominated by incoming data rate, not by the widgets Lean touches.

Root cause confirmed: the QRhiWidget compositing tax

The panadapter is a GPU QRhiWidget; the meters/VFO/scope are software QWidgets in the same window. Every frame, Qt's QBackingStoreDefaultCompositor takes the full-window software backing store, uploads it to a GPU texture (toTexture → blend_untransformed_argb + Metal copyFromBuffer:toTexture:), and composites it with the QRhiWidget's native texture. That full-window upload is the bulk of the pooled CPU (~0.8 cores), it's pixel-area-bound (hence far worse on 5K), and it fires whenever either the panadapter presents a new frame or any software widget repaints.

Critical measurement caveat: the panadapter present rate scales with how much FFT/waterfall data the radio is sending, so idle pooled CPU swings wildly with band activity (we measured the same UI state at anywhere from 0 to ~1040 ms/s pooled across quiet vs busy bands). Single idle captures are therefore unreliable for attributing pooled cost — only same-session, same-band pairs are trustworthy.

Matched same-session A/B (the trustworthy numbers)

Lean OFF vs Lean ON, identical band, meters open, idle:

	Total	Main thread	Pooled
Lean OFF	1.62 cores	606 ms/s	890 ms/s
Lean ON	1.65 cores	477 ms/s (−21%)	1043 ms/s

→ Main-thread −21%; total CPU flat (pool difference is within data-rate noise). The main-thread reduction is the reproducible, felt "zippier" (input/drag run on the main thread).

Meter contribution, same session (Lean on, throttle active), meters open vs collapsed:

	Total	Main	Pooled
Meters open	1.65	477	1043
Meters collapsed	1.22	311	785

→ Open meters add ~0.43 cores even with the 12 Hz throttle. The throttle cuts meter paint rate (S-meter 9.5→6.6/s) but each surviving repaint still triggers a full-window upload, so the residual is significant.

The journey (cross-session — directional only, NOT controlled for band activity)

State	Total	Main	Pooled	Notes
Baseline	1.69	682	871	original idle
Lean (button only)	1.36	430	830
Lean + meters closed	1.17	317	761
Lean 30 Hz + meters closed	0.15	61	0	quiet band — variance, not a real 10× win
Lean 30 Hz + meters open	1.38	387	858
Lean + native applet panel	1.28	327	834	native-layering did not isolate meters under Qt 6.11/macOS — reverted
Lean + meter throttle (open)	1.65	477	1043	busier band

The "0.15 cores" line was initially over-read as a Lean win; the matched A/B later showed it was largely a quiet band.

Changes implemented (branch feat/lean-display)

A checkable Lean button in SpectrumOverlayMenu → MainWindow::applyLeanMode(bool), persisted as LeanMode in AppSettings, applied across all panes/VFOs, reversible, restored on startup:

1. SpectrumWidget::setLeanMode — render an opaque single layer (skip the wallpaper texture in both GPU + software paths), drop the translucent FFT fill, and cap spectrum/waterfall repaint (currently 30 Hz).
2. VfoWidget::setOpaqueMode — swap the #VfoWidgetRoot stylesheet from background: transparent to opaque so the panel composites as a cacheable opaque layer (the stylesheet — not just WA_TranslucentBackground — governs Qt's opacity decision here).
3. WaveApplet::setActive — hide the WAVE scope and drop its sample feed so it never repaints.
4. MeterSmoother::shouldRepaint() — shared static gate; meters keep integrating ballistics at 120 Hz but repaint is gated to ~12 Hz and batched across all meters. Wired into the comp/gate/de-ess GR meters and the S-meter needle.

What didn't work / honest limits

• Native-layering the applet panel (WA_NativeWindow) to isolate meter repaints: no effect under Qt 6.11/macOS (pooled 858→834). Reverted.
• Meter throttle: reduces meter paint rate and helps the main thread, but does not neutralize the pooled cost — 12 Hz × several meters still drives full-window uploads (~0.26 cores residual).
• The 30 Hz spectrum cap: marginal once meters are open; the panadapter present rate is data-driven and dominates.

Where the real pooled win has to come from

The pooled tax is structural: a full-window software-backing-store→GPU-texture upload per composite. The only levers that can actually cut it:

1. Dirty-region upload — confirm/force Qt to upload only the changed rect, not the full window (would make the 5K penalty largely vanish). Highest-value investigation.
2. Move the software UI (meters/VFO) onto the GPU so there's no software backing store to re-upload — large effort.

Net: keep Lean for the main-thread / responsiveness win it reliably delivers; the pooled compositing tax needs the dirty-region or GPU-UI fix, tracked separately.