Neural Network Training as an Optimal Control Problem : — An Augmented Lagrangian Approach —
Neural Network Training as an Optimal Control Problem : — An Augmented Lagrangian Approach —
Training of neural networks amounts to nonconvex optimization problems that are typically solved by using backpropagation and (variants of) stochastic gradient descent. In this work, we propose an alternative approach by viewing the training task as a nonlinear optimal control problem. Under this lens, backpropagation amounts to the sequential approach …