DEJA-VU: Double Feature Presentation and Iterated Loss in Deep Transformer Networks
DEJA-VU: Double Feature Presentation and Iterated Loss in Deep Transformer Networks
Deep acoustic models typically receive features in the first layer of the network, and process increasingly abstract representations in the subsequent layers. Here, we propose to feed the input features at multiple depths in the acoustic model. As our motivation is to allow acoustic models to re-examine their input features …