Penalty Decoding: Well Suppress the Self-Reinforcement Effect in Open-Ended Text Generation
Penalty Decoding: Well Suppress the Self-Reinforcement Effect in Open-Ended Text Generation
The decoding algorithm is critical for open-ended text generation, transforming latent representations into coherent and meaningful outputs. This paper investigates the self-reinforcement effect in text generation and the effectiveness of a repetition penalty to mitigate it. However, determining the optimal repetition penalty value is challenging. To tackle this, we propose …