confusing contact_head in RNABertModel

[xzdlj]

Thank you very much for open-sourcing RNAFM!

I have a question regarding the function _init_submodules_common, which is called when initializing the RNABertModel. In this function, there is a contact_head used for contact prediction, which requires related labels when computing the loss. However, as mentioned in the paper, the pretraining dataset does not include labels for contact prediction. This is quite confusing to me.

Could you clarify whether the contact_head is actually used during pretraining?

If I have misunderstood or missed anything, please feel free to point it out.