You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I searched some code in longformer and the related code in transformers,the invert_mask() function in BartEncoder destroies the integer 2 in the attention mask,but the longformer attention code regard the mask as it dosen't been inverted.
So I think the global attention is not enabled in the model, could you explain it to me ?
Hope that I'm wrong....
The text was updated successfully, but these errors were encountered:
I searched some code in longformer and the related code in transformers,the invert_mask() function in BartEncoder destroies the integer 2 in the attention mask,but the longformer attention code regard the mask as it dosen't been inverted.
So I think the global attention is not enabled in the model, could you explain it to me ?
Hope that I'm wrong....
The text was updated successfully, but these errors were encountered: