Skip to content

perf(ptl): replay independent CUDA segmentation predictions - #22

Merged
Borda merged 11 commits into
Borda:mainfrom
JESUSROYETH:codex/segmentation-prediction-replay
Oct 20, 2026
Merged

Borda merged 11 commits into
Borda:mainfrom
JESUSROYETH:codex/segmentation-prediction-replay

Conversation

@JESUSROYETH

@JESUSROYETH JESUSROYETH commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

--model.capture_segment_heads true replays the segmentation head's existing prediction calls and its first-order backward instead of re-running them. The native arithmetic and the unused-parameter gradient boundaries stay exactly as they are. It's off by default. Coverage is narrow too: one CUDA device, BF16 mixed precision, torch 2.9+, automatic optimisation, no gradient accumulation.

I ran five alternating native/replay pairs on an isolated g2-standard-8 NVIDIA L4, against main e4db46d:

Full run Median (range), s
Native 202.794 (201.536–204.709)
Replay 169.130 (168.167–169.786)

So the full run takes 16.60% less time on COCO val2017 (384 training / 32 disjoint validation images, 80 classes, nano, batch 4, 640 px, 10 epochs). That number covers the whole run: construction, caching, recording, training, EMA, validation and checkpoints. Both arms already had prepared caching, device HSV, epoch prefetch and orthogonalisation/update replay turned on, so the gain sits on top of those. Torch 2.9.1+cu129, driver 580.178.04, Lightning 2.6.6. Whole-process wall time falls 16.16%, and peak reserved CUDA memory goes up a bit, 6084→6474 MiB.

Model/optimiser/scheduler/EMA/loss byte hashes and the validation metrics match across the five pairs, and a separate COCO prediction check matches raw bytes too. Full pre-commit passes, so does the offline make test: 4343 tests, 96.43% coverage against the unchanged 96% floor. The focused recorder suite on the L4 reports 134 passed; on torch 2.4 it reports 126 passed and 8 skipped. A real Trainer regression checks that graph launches stay correct after EMA validation. What I didn't run this round: the full make gate, the accelerator goldens, and a long-run accuracy check. Those stay unverified.

That gain is limited to this batch-4 workload, though. An earlier single batch-16 screen only gave 2.01%, and I haven't re-checked larger-batch performance against current source yet.

Clean-room acknowledged: native expressions, R43 and our measurements; no method arithmetic or frozen value changes.

WP: 193e
Provenance: R43; native prediction calls and measured CUDA profiles
Assumptions: none
Gate: tests/ptl/test_segment_head_capture.py

WP: 193e
Provenance: R43; native prediction calls and measured CUDA profiles
Assumptions: none
Gate: tests/ptl/test_segment_head_capture.py

---

Signed-off-by: Jesús Royeth <JESUSROYETH@users.noreply.github.com>
@JESUSROYETH
JESUSROYETH requested a review from Borda as a code owner October 20, 2026 10:48
@kilo-code-bot

kilo-code-bot Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (13 files)
  • CHANGELOG.md
  • docs/ROADMAP.md
  • docs/TRAINING.md
  • docs/engineering/phase-17-pose-objective-parity.md
  • docs/work_packages/wp-193c.md
  • docs/work_packages/wp-193d.md
  • docs/work_packages/wp-193e.md
  • docs/work_packages/wp-194.md
  • src/lucid_yolo/models/heads/detect.py
  • src/lucid_yolo/ptl/module.py
  • src/lucid_yolo/ptl/segment_head_capture.py
  • tests/ptl/test_segment_head_capture.py
  • tests/ptl/test_segment_head_capture_guards.py
Previous Review Summary

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review

This review did not finish. The model reached its output limit before it
could write the review — a reasoning model can spend the whole budget thinking.
Re-run the review, or lower the model's thinking effort, and it should get
further. Any inline comments below are from an earlier review.


Reviewed by free · Input: 57.5K · Output: 2.4K · Cached: 385.3K

@Borda Borda added the enhancement New feature or request label Oct 20, 2026
@Borda
Borda requested a balanced review from Copilot October 20, 2026 12:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

CUDA forward/backward replay warrants human review, with the full gate, accelerator goldens and long-run accuracy checks still outstanding.

No open issues 🎉

What changed in this PR

Adds opt-in CUDA replay for segmentation predictions and their first-order backward work, preserving native arithmetic and unused-mask gradient behavior.

Changes:

  • Introduces guarded BF16 prediction replay with fit-scoped cleanup and native fallback.
  • Adds regression tests for exact outputs, gradients, compatibility and lifecycle behavior.
  • Documents configuration, measured performance, memory costs and validation limits.
File Description
tests/​ptl/​test_segment_head_capture.py Tests replay correctness, guards and cleanup.
src/​lucid_yolo/​ptl/​segment_head_capture.py Implements prediction recording and replay.
src/​lucid_yolo/​ptl/​module.py Integrates the option and lifecycle callbacks.
src/​lucid_yolo/​models/​heads/​detect.py Adds prediction scheduling without changing arithmetic.
docs/​work_packages/​wp-194.md Updates work-package navigation.
docs/​work_packages/​wp-193e.md Records implementation, measurements and validation gaps.
docs/​work_packages/​wp-193c.md Links the new dependent work package.
docs/​TRAINING.md Documents usage, requirements and limitations.
docs/​ROADMAP.md Tracks WP-193e and outstanding validation.
docs/​engineering/​phase-17-pose-objective-parity.md Includes the new engineering record.
CHANGELOG.md Announces the opt-in feature.

Review effort: Balanced

JESUSROYETH and others added 10 commits October 20, 2026 09:35
Retain the existing segmentation prediction replay and current detection
loss replay callbacks when merging the latest main.

WP: 193e
Provenance: R43; existing callback composition and native prediction calls
Assumptions: none
Gate: tests/ptl/test_segment_head_capture.py
Signed-off-by: Jesús Royeth <JESUSROYETH@users.noreply.github.com>
Exercise buffer restoration, layout certification and replay lifecycle
composition on CPU while retaining the real CUDA regression cases.

WP: none
Provenance: R43; existing host-side prediction replay contracts
Assumptions: none
Gate: tests/ptl/test_segment_head_capture_guards.py
Signed-off-by: Jesús Royeth <JESUSROYETH@users.noreply.github.com>
Retain merged pose replay setup alongside segmentation prediction replay.
Exercise a real automatic fit with EMA, instance-mask validation and CUDA
graph launches after validation, while retaining the CPU coverage floor.

WP: 193e
Provenance: R43; native replay contracts and own CUDA integration measurements
Assumptions: none
Gate: tests/ptl/test_segment_head_capture.py
Signed-off-by: Jesús Royeth <JESUSROYETH@users.noreply.github.com>
Record the latest-main complete-fit measurements separately from the earlier
cohorts, and describe the real Trainer regression and its backend checks.

WP: 193e
Provenance: R43; native replay contracts and own CUDA measurements
Assumptions: none
Gate: tests/ptl/test_segment_head_capture.py
Signed-off-by: Jesús Royeth <JESUSROYETH@users.noreply.github.com>
- CHANGELOG.md: keep both Added entries, WP-193e placed after WP-193d
- docs/ROADMAP.md: keep main's WP-193d and WP-199 rows, add the WP-193e row after WP-193d
- phase-17-pose-objective-parity.md: keep both includes, WP-193d then WP-193e
- wp-193c.md, wp-194.md, wp-193d.md, wp-193e.md: navigation and needed-by lines follow the 193c, 193d, 193e, 194 row order (wp-193d.md and wp-193e.md regenerated by audit_wp_links.py --fix)
- module.py: keep main's pose head topology and loss-capture lifecycle beside the segment-head capture; _init_segmentation replaces main's inline segmentation stage build at the same point, so construction order is unchanged

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Signed-off-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Retain the earlier batch-16 counter-screen as historical evidence, state the
current larger-batch gap, and explain the private prediction dispatch seam.

WP: 193e
Provenance: R43; native call boundaries and own CUDA measurements
Assumptions: none
Gate: tests/ptl/test_segment_head_capture.py
Signed-off-by: Jesús Royeth <JESUSROYETH@users.noreply.github.com>
Preserve the newly published main merge and its changelog order. Production
source matches the measured cohort; retain the reviewed tests and evidence.

WP: 193e
Provenance: R43; native repository history and own CUDA measurements
Assumptions: none
Gate: tests/ptl/test_segment_head_capture.py
Signed-off-by: Jesús Royeth <JESUSROYETH@users.noreply.github.com>
- Validate `capture_segment_heads` in `_validate_replay_options` beside the sibling replay flags, refusing a non-segment task or `compile_step=True` there instead of inside `_init_segmentation`, and document why the compiler is refused while the segment objective stays eager.
- Pass the prediction runner straight through `forward_branch` and `_base_predictions`, dropping the `_SegmentHeadCapture.base` indirection; `_runner` docstrings mark the seam private.
- Keep an empty batch on the native call without retiring replay for the fit, and drop the empty-tensor clause from the feature breach.
- Name the unlisted stage type in a new `_layer_breach` instead of reporting it as a forward override, and give the precision-policy retire reason its context.
- Align the warning wording with the sibling replay flags (`stays eager`, `is ignored outside ...`).
- State in the holder docstring that replayed coefficient, prototype and semantic outputs are static graph tensors overwritten by the next replay, and that one input signature is recorded per group per fit.
- Comment the two coefficient-path asserts with the invariant that makes them unreachable.

[resolve group] PR Borda#22 — items 27 28 30 8 12 13 14 9 7

WP: 193e
Provenance: none (review remediation of the replay option surface and guard messages; repository code only)
Assumptions: none
Gate: tests/ptl/test_segment_head_capture.py, tests/ptl/test_segment_head_capture_guards.py, tests/models/test_head.py

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Signed-off-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
- Add CPU tests for runner refusal wiring (head dispatch, CPU features), the unreadable precision policy message, an output without a backward node never holding, flag-on versus flag-off train forward and gradients, a real CPU `Trainer.fit` smoke that warns once without replay, local and global stage hook causes, an empty batch staying native without retiring, and the unlisted stage type message.
- Leave out cases the contributor guard suite already asserts on the same path.
- Route the guard suite's base-prediction row through `_base_predictions(features, _runner=holder)` now that the holder no longer exposes `base`.
- Drop articles from test names, give dict and dtype parametrize cases readable ids, drop the `callable(test_...)` examples on test functions, and match the `stays eager` warning wording.

[resolve group] PR Borda#22 — items 3 15 18 16 2 19 26 30

WP: 193e
Provenance: none (review remediation adding tests for existing guards; repository code only)
Assumptions: none
Gate: tests/ptl/test_segment_head_capture.py, tests/ptl/test_segment_head_capture_guards.py

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Signed-off-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
- Scope byte-exact agreement to NVIDIA L4 (sm_89) with torch 2.9.1+cu129 in TRAINING.md and the changelog entry; other GPUs and torch builds are unverified.
- Say why gradient accumulation stays native (validated only at `accumulate_grad_batches=1`, no known defect) in TRAINING.md and WP-193e.
- Document in TRAINING.md that replayed coefficient, prototype and semantic outputs are overwritten by the next replay and that other input signatures run native while graph pools stay reserved until the fit ends.
- State that the runtime source digest names neither its hashed artifact nor its hashing method.
- Use the sibling work-package heading form for WP-193e.

[resolve group] PR Borda#22 — items 25 23 29 24 9 7

WP: 193e
Provenance: none (review remediation scoping documented claims; repository documents only)
Assumptions: none
Gate: pre-commit run --all-files

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Signed-off-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
@Borda
Borda merged commit b293970 into Borda:main Oct 20, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants