Implement BoN for training and eval#528

Open

Dahoas wants to merge 40 commits into

mainfrom

Commits on Jul 18, 2023

Implementing support for dense rewards
Dahoas
committed
added "num_return_sequences" param which corresponds to n in Best-of-N sampling

SharathRaparthy
authored and
Dahoas
committed
updates to "num_return_sequences" param

SharathRaparthy
authored and
Dahoas
committed
BoN implementation

SharathRaparthy
authored and
Dahoas
committed
Changed back to default.

SharathRaparthy
authored and
Dahoas
committed
TopK sampling instead of Top1

SharathRaparthy
authored and
Dahoas
committed
summed along dim=1

SharathRaparthy
authored and
Dahoas
committed
Generating samples in chunks

SharathRaparthy
authored and
Dahoas
committed
added gen_chunk_size parameter

SharathRaparthy
authored and
Dahoas
committed
chunking in forward prop

SharathRaparthy
authored and
Dahoas
committed
chunking generations in train and eval

SharathRaparthy
authored and
Dahoas
committed
Implementing support for dense rewards
Dahoas
committed
Fix distributed ref_mean, ref_var bug for dense rewards
Dahoas
committed
Make generation respect max seq length
Dahoas
committed
Make experience before first round of training
Dahoas
committed
Refactoring .generate/.generate_eval
Dahoas
committed
Fix BoN metric support
Dahoas
committed
Enforce chunk_size param for eval generation when present
Dahoas
committed
Fix: Don't shuffle prompt dataset
Dahoas
committed
Move inputs to device
Dahoas
committed
Fix style
Dahoas
committed

Commits on Jul 21, 2023

Fix chunked generation
Dahoas
committed

Commits on Jul 22, 2023

fix(accelerate_base_trainer): order of keyword arguments
maxreciprocate
authored

Commits on Aug 21, 2023

Commits on Sep 1, 2023