New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated BART to Longformer-encoder-decoder (LED) converter #245

Open

erichans opened this issue Oct 21, 2022 · 0 comments

erichans commented Oct 21, 2022

Hi @ibeltagy et al., I'm pre-training BART to Portuguese and converting the pre-trained model to LED following the instructions you gave in the paper and the code at https://github.com/allenai/longformer/blob/caefee668e39cacdece7dd603a0bebf24df6d8ca/scripts/convert_bart_to_longformerencoderdecoder.py.

The huggingface library is evolving fast; unfortunately, the code you provided is outdated and I had to implement a new version based on yours.

I have 2 questions:

Could you tell me if everything is ok or if I missed something? https://gist.github.com/erichans/af745a381b28b1c019f96997ddac4cd7
Is the LEDForConditionalGeneration model uploaded to huggingface just a BART model converted to LED or is there something else?

Thanks in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment