CLS token in SSL

[980202006]

Hello, this paper is very ingenious, solving many gaps in SSL and generative training. I have a question: Why use CLS tokens instead of patch-level loss during SSL training?

[jiachenlei]

Hi, thanks for your interest in our work.

• For representation learning, applying a contrastive loss at the patch level is non-trivial because positive and negative pairs are constructed via data augmentation. Spatial augmentations (like random cropping) disrupt spatial alignment, making patch-to-patch correspondence ill-defined.

• EPG Representation Alignment: Even though no data augmentation is used for the alignment, a patch-level loss would introduce a scale mismatch relative to the representation learning term, complicating hyper-parameter tuning.