Question- 2 extra embeddings

[danaru29]

When I give the model a sequence I get back a shape of [L + 2, 640]
where L is the length of the sequence. How can I match the embeddings to the actual sequence length?

Thanks!

[TaykhoomDalal]

In the tokenization function (RNA-FM/fm/data.py) they prepend a beginning of sequence token (bos) and append an end of sequence token (eos) which is why its L+2. You can ignore the first and last column embeddings to match it to the actual sequence length.