MimCo: Masked Image Modeling Pre-training with Contrastive Teacher
MimCo: Masked Image Modeling Pre-training with Contrastive Teacher
Recent masked image modeling (MIM) has received much attention in self-supervised learning (SSL), which requires the target model to recover the masked part of the input image. Although MIM-based pre-training methods achieve new state-of-the-art performance when transferred to many downstream tasks, the visualizations show that the learned representations are less …