Skip to content

feature: Improving architectures #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
AlexandreAdam merged 74 commits into from
Nov 23, 2024
Merged

feature: Improving architectures #13

AlexandreAdam merged 74 commits into from
Nov 23, 2024

Conversation

@AlexandreAdam
Copy link
Owner

@AlexandreAdam AlexandreAdam commented Nov 22, 2024

Change Log

  • Added EDM Score Model.
  • Added EDMv2Net.
  • Refactored some arguments in neural nets.
  • Updated default Fourier scale to 0.02 instead of 16.
  • Added Inverse square root decay for the learning rate, set to constant LR by default, in the trainer.
  • Added Karras EMA, with logic to save checkpoints according to multiple ema_lengths.
  • Added PostHoc EMA wrapper to synthesize arbitrary neural network profile from saved checkpoints, especially when multiple lengths profile are present, in which case this wrapper is used to synthesize a model at the required ema_length.
  • Included soft reset in EMA, by default is turned off.

@AlexandreAdam
Removed dollar signs in README
8a098b1
@AlexandreAdam
Updated actions
e0a6819
@AlexandreAdam
Update CI action to run on any branch push
dbf46c5
@AlexandreAdam
Fixed bug in CI action
17396f4
@AlexandreAdam
Modved package to src to avoid conflict
99211f6
@AlexandreAdam
Reduced memory reqs on lora sbm test for macOS backend
ed7a0b5
@AlexandreAdam
Added restriction for sending the coverage
6c621a9
@AlexandreAdam
Removed macOS backend test and updated flag for coverage
d310efe
@AlexandreAdam
Fixed CI action bug
3934a1b
@AlexandreAdam
Removed flag nonsense in CI action
be4c1f4
@AlexandreAdam
Removed pip show in CI
1afa678
@AlexandreAdam
Adding docs folder, currently empty
19d670f
@AlexandreAdam
Started docs, drafted a style and created intro page
7da4626
@AlexandreAdam
Modified docs action and added utils for plotting and distributions
860a8bb
@AlexandreAdam
Updated book
9923113
@AlexandreAdam
Improved introduction
48121fc
@AlexandreAdam
Improved intro
97824a3
@AlexandreAdam
Renamed intro to the score
1571eba
@AlexandreAdam
Added readthedocs conf file
f70170f
@AlexandreAdam
Updated actions
dc27335
@AlexandreAdam
Updated permission of github action
d67fe55
@AlexandreAdam
updated readthedocs conf
1792c80
@AlexandreAdam
Reset readthedocs file
f71e671
@AlexandreAdam
Added documentation badge
881cfd5
@AlexandreAdam
Added documentation section
03da72d
@AlexandreAdam
Drafted some structure for the docs
9b26a7d
@AlexandreAdam
Started the score matching section
3db11b1
@AlexandreAdam
Added part on annealing
fc1c0e2
@AlexandreAdam
Finished annealing score matching section
9f0e1a7
@AlexandreAdam
Edited score matching part
fd6032e
@AlexandreAdam
Added docstring for dlogp in ODE Solver class
45323f9
@AlexandreAdam
Joint training second order
e321ff2
@AlexandreAdam
Started working on EDM parametrization
2bba9a5
@AlexandreAdam
Drafted new edm parametrization
45e6384
@AlexandreAdam
Drafted edm stuff
81261f1
@AlexandreAdam
Merged dev
dc49fba
@AlexandreAdam
Updated the overview section
bf55a8a
@AlexandreAdam
Merged dev
f66c76f
@AlexandreAdam
Merge branch 'docs' into improving_architectures
4b3df52
@AlexandreAdam
Renamed solvers, updated SDE to have the skip connections needed for …
083585e 
…the EDM setup
@AlexandreAdam
Fixed a bug in Solver constructor
1367dd4
@AlexandreAdam
Updated hyperparameters of some SDE to make sure they work nicely wit…
7c57382 
…h EDM
@AlexandreAdam
Removed Tweedie from solvers, brought it back in Score Model class
ad9482f
@AlexandreAdam
Added the sampling distribution for EDM
aff95e0
@AlexandreAdam
Added functionality to automatically load EDM model from path
fd8d57d
@AlexandreAdam
Updated code for EDMv2Net and tested it in the API
08a0e42
@AlexandreAdam
Added inverse square root learning rate scheduler, tested the saving/…
4ea508e 
…loading logic so that global step can be reloaded from a checkpoint
@AlexandreAdam
Made sure analytical score models would throw error upon trying to sa…
bb1befc 
…ve or load
@AlexandreAdam
Working on logic of loading checkpoints with different ema_lengths
882b9fa
@AlexandreAdam
Implemented the PostHocEMA wrapper, modified the loading logic so it …
8f39311 
…could handle the case where multiple ema_lengths are present
@AlexandreAdam
Minor changes, preparing for futur updates
5eda2e0
@AlexandreAdam
Added my own EMA to control better the soft reset and the various par…
9a81dc8 
…ametrization, now Karras EMA is back a default for the trainer as the preferred method.
@AlexandreAdam
Debugged soft reset
c9c61be
@AlexandreAdam AlexandreAdam changed the title feat: Improving architectures feature: Improving architectures Nov 23, 2024
@AlexandreAdam AlexandreAdam merged commit 15de2ed into dev Nov 23, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@AlexandreAdam