Code

BatchNorm training instability fix#675

Code

Open

andrewdipper wants to merge 6 commits intopatrick-kidger:mainpatrick-kidger/equinox:mainfrom andrewdipper:mainandrewdipper/equinox:mainCopy head branch name to clipboard

+204-28

Commits

Commits on Mar 7, 2024

• Update BatchNorm to add mode for using batch statisics / increase stability when using running statistics.

  

  andrewdipper

  committed
  346e472

  Copy full SHA for 346e472

• fix batchnorm type definiton

  

  andrewdipper

  committed
  cc021f6

  Copy full SHA for cc021f6

Commits on Mar 14, 2024

• use unbiased variance + minor fixes

  

  andrewdipper

  committed
  6bfa77f

  Copy full SHA for 6bfa77f

• reformat + comments

  

  andrewdipper

  committed
  e7705b6

  Copy full SHA for e7705b6

Commits on Apr 1, 2024

• add BatchNorm backward compatibility mode

  

  andrewdipper

  committed
  7a99c7f

  Copy full SHA for 7a99c7f

• fix description

  

  andrewdipper

  committed
  96f4a96

  Copy full SHA for 96f4a96