Molecule-Space: Free Lunch in Unified Multimodal Space via Knowledge
Fusion
Molecule-Space: Free Lunch in Unified Multimodal Space via Knowledge
Fusion
Unified multi-model representation spaces are the foundation of multimodal understanding and generation. However, the billions of model parameters and catastrophic forgetting problems make it challenging to further enhance pre-trained unified spaces. In this work, we propose Molecule-Space, an idea that treats multimodal representation spaces as "molecules", and augments pre-trained unified …