Prevalence of neural collapse during the terminal phase of deep learning training
Prevalence of neural collapse during the terminal phase of deep learning training
Modern practice for training classification deepnets involves a Terminal Phase of Training (TPT), which begins at the epoch where training error first vanishes; During TPT, the training error stays effectively zero while training loss is pushed towards zero. Direct measurements of TPT, for three prototypical deepnet architectures and across seven …