Skip to content

Stabilize ChaosGrad defaults and preserve trainer/optimizer API compatibility#9

Merged
theomgdev merged 1 commit intoexperimentalfrom
copilot/sub-pr-8
Mar 18, 2026
Merged

Stabilize ChaosGrad defaults and preserve trainer/optimizer API compatibility#9
theomgdev merged 1 commit intoexperimentalfrom
copilot/sub-pr-8

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 18, 2026

RealNet training stack needed safer defaults and backward-compatible APIs after expanding ChaosGrad and trainer configurability.

  • API compatibility: Kept new max_grad_norm and ChaosGrad hyperparameters keyword-only/append-only to avoid breaking positional callers; documented new args.
  • Optimizer behavior: Tuned plateau noise scaling to rely on gradient magnitude rather than near-zero params; clarified loss_history_min semantics to match retained-history behavior.
  • Trainer robustness: Synced gradient persistence with AMP scaling and exposed configurable gradient clipping without shifting existing positional arguments.

Example:

optimizer = ChaosGrad(
    model.parameters(),
    lr=3e-4,
    adaptive_ema=0.2,
    plateau_noise_intensity=0.5,
    loss_history_min=64,        # retained history floor
    sentinel_threshold=1e-5,    # input health threshold
)

trainer = RealNetTrainer(
    model, data, optimizer,
    max_grad_norm=1.0  # keyword-only, preserves positional API
)

📱 Kick off Copilot coding agent tasks wherever you are with GitHub Mobile, available on iOS and Android.

Copilot AI changed the title [WIP] Fix critical architecture bugs and refactor ChaosGrad and Trainer Stabilize ChaosGrad defaults and preserve trainer/optimizer API compatibility Mar 18, 2026
Copilot AI requested a review from theomgdev March 18, 2026 00:24
@theomgdev theomgdev marked this pull request as ready for review March 18, 2026 00:28
Copilot AI review requested due to automatic review settings March 18, 2026 00:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@theomgdev theomgdev merged commit f7ba0f3 into experimental Mar 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants