Directory for training components of PRAG is mostly self-contained.
- Guided generator is the inference directory, with dependency on trained personalized retriever and embedding estimator.
- The implementation is losely Huggingface🤗-like (i.e. they will look similar to other pipelines using the HF ecosystem).
- We recommend an 24-GB GPU for training PRAG (tuning UnifiedQA component requires 48GB, but it should finish in minutes).
- Noisy_sum directory is used to train distantly supervised summarizer baseline, and coop-finetune is for Optimus based sumarizer.
- You can find the appendix here.
Some modules in this repository is based-off publically available implementation of prior models. This includes Optimus implementation by Megagon labs, and a series of models released by Lei Li. The dataset we used could be found here, also made available by Lei Li (kudos to previous works for open-sourcing!).