Prefer a chat interface with context about you and your work?
On the diffusion approximation of nonconvex stochastic gradient descent
We study the Stochastic Gradient Descent (SGD) method in nonconvex optimization problems from the point of view of approximating diffusion processes. We prove rigorously that the diffusion process can approximate the SGD algorithm weakly using the weak form of master equation for probability evolution. In the small step size regime …