Prefer a chat interface with context about you and your work?
Pipelined Training with Stale Weights in Deep Convolutional Neural Networks
The growth in size and complexity of convolutional neural networks (CNNs) is forcing the partitioning of a network across multiple accelerators during training and pipelining of backpropagation computations over these accelerators. Pipelining results in the use of stale weights. Existing approaches to pipelined training avoid or limit the use of …