Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks
Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks
Although large language models have exhibited impressive zero-shot ability, the huge model size generally incurs high cost. Recently, semi-parametric language models, which augment a smaller language model with retrieved related background knowledge, alleviate the need for storing everything into the model parameters.Although existing semi-parametric language models have demonstrated promising language …