Skip to content

perf(ptl): replay shared CUDA segmentation features - #25

Merged
Borda merged 8 commits into
Borda:mainfrom
JESUSROYETH:codex/segment-shared-feature-replay
Oct 21, 2026
Merged

Borda merged 8 commits into
Borda:mainfrom
JESUSROYETH:codex/segment-shared-feature-replay

Conversation

@JESUSROYETH

Copy link
Copy Markdown
Collaborator

--model.capture_network true now replays the backbone and the neck for segmentation too, and it can be combined with capture_segment_heads — the prediction groups keep independent gradients and the native loss arithmetic. The flag is off by default, and accumulation or an altered head/forward dispatch falls back to the native path.

I ran five alternating pairs on an isolated L4, against main (b293970), with the existing training options on: nano segmentation, batch 4, 640 px, BF16, five epochs, 512 COCO val2017 images for training and 16 disjoint images for validation. Complete-fit time dropped 18.42%:

Seconds, median (range) Native features Replay
Complete fit 107.412 (106.764–109.816) 87.629 (87.043–88.462)
Whole process 112.721 (111.918–115.208) 92.937 (92.471–93.827)

Model, optimizer, scheduler, EMA and per-step-loss bytes match across the ten fits. Peak reserved CUDA memory rises from 5,496 to 5,702 MiB. Focused CPU/CUDA regressions and the full pre-commit, strict mypy included, pass.

I also screened batch 16 once and it only saved 2.90%, so the claim above stays at batch 4. Full make gate, the accelerator goldens and long-run accuracy are still unverified; the measured scope is written up in WP-193g.

Clean-room acknowledged: this only touches native stage composition, our own profiles and the registered PyTorch API R43 — no new numerical choice.

WP: 193g
Provenance: R43; native shared stages and measured CUDA profiles
Assumptions: none
Gate: tests/ptl/test_segment_network_capture.py

WP: 193g
Provenance: R43; native shared stages and measured CUDA profiles
Assumptions: none
Gate: tests/ptl/test_segment_network_capture.py

---

Signed-off-by: Jesús Royeth <JESUSROYETH@users.noreply.github.com>
@JESUSROYETH
JESUSROYETH requested a review from Borda as a code owner October 20, 2026 23:09
@kilo-code-bot

kilo-code-bot Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (12 files)
  • CHANGELOG.md — added WP-193g entry; consistent with existing entries
  • docs/ROADMAP.md — added WP-193g row; dependency links updated in 193d/193e/194
  • docs/TRAINING.md — segmentation capture_network paragraph added; task list extended to include task: segment
  • docs/engineering/phase-17-pose-objective-parity.md — WP-193g include added in phase order
  • docs/work_packages/wp-193d.md — "Needed by" updated to WP-193g
  • docs/work_packages/wp-193e.md — "Needed by" and navigation updated to WP-193g
  • docs/work_packages/wp-193g.md — new work-package log
  • docs/work_packages/wp-194.md — navigation updated
  • src/lucid_yolo/ptl/module.py — _segmentation_features, _EAGER_SEGMENT_FORWARD, topology capture, validation and docstrings
  • src/lucid_yolo/ptl/network_capture.py — docstring and run_features retirement message generalised from "pose" to "shared network"
  • tests/ptl/test_network_capture.py — parametrised cases and error regex updated
  • tests/ptl/test_pose_network_capture.py — retirement warning text updated
  • tests/ptl/test_segment_network_capture.py — new test module

Reviewed by free · Input: 314.4K · Output: 46.8K · Cached: 779.8K

@Borda
Borda requested a balanced review from Copilot October 20, 2026 23:28
@Borda Borda added the enhancement New feature or request label Oct 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

CUDA backward replay needs human review while full-gate, accelerator-golden and long-run accuracy validation remain unverified.

No open issues 🎉

What changed in this PR

Extends opt-in CUDA network replay to segmentation’s backbone and neck while preserving independent prediction gradients and native loss computation.

Changes:

  • Adds segmentation replay routing, compatibility checks and native fallbacks.
  • Adds parity, lifecycle, fallback and CLI regression tests.
  • Documents usage, measured performance and remaining validation gaps.
File Description
tests/​ptl/​test_segment_network_capture.py Adds segmentation replay regression coverage.
tests/​ptl/​test_pose_network_capture.py Updates the frozen-feature warning assertion.
tests/​ptl/​test_network_capture.py Updates option compatibility expectations.
src/​lucid_yolo/​ptl/​network_capture.py Generalizes shared-feature documentation and warnings.
src/​lucid_yolo/​ptl/​module.py Enables guarded segmentation feature replay.
docs/​work_packages/​wp-194.md Updates work-package navigation.
docs/​work_packages/​wp-193g.md Records implementation, benchmarks and validation limits.
docs/​work_packages/​wp-193e.md Links the new dependent work package.
docs/​work_packages/​wp-193d.md Links the new dependent work package.
docs/​TRAINING.md Documents segmentation replay usage and constraints.
docs/​ROADMAP.md Adds WP-193g and its validation status.
docs/​engineering/​phase-17-pose-objective-parity.md Includes the new engineering record.
CHANGELOG.md Announces segmentation feature replay.

Review effort: Balanced

Borda and others added 7 commits October 21, 2026 12:28
…feature-replay

Brings in oriented-network replay and pose prediction-stem replay. Replay validation now accepts detect, keypoints, obb and segment; the exact-message replay-option test keeps main's anchored form with the merged message. Work-package rows, navigation and Needed-by lines order WP-193f before WP-193g.

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Signed-off-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
- WP-193g page and roadmap row: "full gate waived" / "explicitly waived" become "not run", since no waiver is on record; offline CI green is claimed for PR head 8ebf10e only, not for the merged tree.
- WP-193g page: the recorded module.py SHA-256 is qualified as the hash of the measured PR-head tree (8ebf10e), which later commits change.
- `_FeatureNetwork` docstring: optional branches without gradients now cover segmentation (coefficient, prototype and semantic predictions) as well as pose point stems.

WP: 193g
Provenance: none (documentation wording only)
Assumptions: none
Gate: wp-links hook; pytest --doctest-modules src/lucid_yolo/ptl/network_capture.py

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Signed-off-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
- `_pose_head_topology` becomes the task-neutral `_head_topology`, held once as `self._head_topology` for keypoints and segmentation instead of two per-task copies.
- New `_head_topology_breach(head, topology, label)` does the topology compare and the hooked-module check; `_pose_head_breach` keeps its pose-only stem dispatch check and delegates the rest, so pose reasons are unchanged.
- Segmentation retires with the specific cause from `_segment_head_breach` or `_head_topology_breach` ("the segmentation head topology or class forward changed", "a segmentation head module is hooked, overridden or compiled") instead of one collapsed message.
- `_network_replay_eligible` holds the activation, training, automatic optimisation, gradient and CUDA-image conditions both training routes repeated; `_retire_accumulated_replay` keys the accumulation retirement on `_ACCUMULATION_EAGER_LABELS`, which leaves detection out as before.
- The `_compiled_call_impl` clause is dropped from the segmentation route: training calls `forward_segmentation` directly, so a compiled module call never reaches it, while the class identity and `vars` checks still catch an overridden or compiled `forward_segmentation`.
- Tests follow the rename and the new reason strings; the "compiled" routing mode no longer expects a retirement.

WP: 193g
Provenance: none (guard restructuring and warning text; no stage arithmetic changed)
Assumptions: none
Gate: tests/ptl/test_segment_network_capture.py::test_segment_host_routing_and_native_fallback, tests/ptl/test_segment_network_capture.py::test_segment_head_hook_retires_feature_replay_and_stays_observable, tests/ptl/test_pose_network_capture.py::test_altered_pose_head_dispatch_fails_certificate

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Signed-off-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
- `_NetworkCapture.retired` exposes whether this fit's recording was retired; `clear()` at the next fit resets it.
- `_network_replay_eligible` returns False once the holder is retired, so later steps of that fit go native without re-walking the head certificate, stem owners or forward identity checks.
- Pose and segmentation tests patch the breach helpers and the recorder and assert none is called after a retirement.

WP: 193g
Provenance: none (control-flow short-circuit; no stage arithmetic changed)
Assumptions: none
Gate: tests/ptl/test_pose_network_capture.py::test_retired_holder_skips_pose_replay_predicates, tests/ptl/test_segment_network_capture.py::test_retired_holder_skips_segment_replay_predicates

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Signed-off-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
- Routing test adds inactive, no-trainer, validating, manual-optimization and cpu-images modes, each expecting no recorder call; a class-level `forward_segmentation` override and a head topology swap get their own tests asserting the retirement reason.
- Routing test runs the real `retire` under `pytest.warns` with a reason pattern per retiring mode, and every other mode with warnings raised as errors.
- Fit parity test counts `_PredictionGroup` recordings for the combined network and head bf16 cell and fails on any "stays eager" warning where replay is expected.
- Fit parity cells: network-only bf16, fp16 and fp32; heads bf16 at accumulation 1 and 2; heads fp16 asserting the head holder's eager warning.
- CUDA lifecycle tests: a step failing after recording releases both holders, a second fit records once per fit, freezing after recording retires once, and raising accumulation after recording retires with the accumulation reason.
- Unused-gradient filter covers `semantic.` and `proto_fusion.` parameters and asserts it matched at least one.
- Replay parity rows drop the heads=True fp32 and fp16 cells, which repeated the heads=False comparison.
- Sticky retirement checks run the later forward with warnings as errors and graph recording forbidden, and assert the holder stays retired.
- The autouse fixture restores `torch.backends.cudnn.benchmark`, which Lightning's trainer setup overwrites.

WP: 193g
Provenance: none (test coverage only)
Assumptions: none
Gate: tests/ptl/test_segment_network_capture.py::test_segment_host_routing_and_native_fallback, tests/ptl/test_segment_network_capture.py::test_segment_head_topology_swap_retires_with_topology_reason, tests/ptl/test_segment_network_capture.py::test_retired_holder_skips_segment_replay_predicates

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Signed-off-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Keep detection, pose, oriented and segmentation replay routes together after
merging the independent network updates. Add constructor regression cases
for all four tasks and retain the complete compiler-refusal message.

WP: 193g
Provenance: R43; existing native replay stages and repository history
Assumptions: none
Gate: tests/ptl/test_network_capture.py

Signed-off-by: Jesús Royeth <JESUSROYETH@users.noreply.github.com>
…e-replay' into codex/segment-shared-feature-replay

---
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

Signed-off-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
@Borda
Borda merged commit 5756ea0 into Borda:main Oct 21, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants