Efficient Adversarial Training with Robust Early-Bird Tickets
Efficient Adversarial Training with Robust Early-Bird Tickets
Adversarial training is one of the most powerful methods to improve the robustness of pre-trained language models (PLMs). However, this approach is typically more expensive than traditional fine-tuning because of the necessity to generate adversarial examples via gradient descent. Delving into the optimization process of adversarial training, we find that …