Learning Gaussian Multi-Index Models with Gradient Flow: Time Complexity
and Directional Convergence
Learning Gaussian Multi-Index Models with Gradient Flow: Time Complexity
and Directional Convergence
This work focuses on the gradient flow dynamics of a neural network model that uses correlation loss to approximate a multi-index function on high-dimensional standard Gaussian data. Specifically, the multi-index function we consider is a sum of neurons $f^*(x) \!=\! \sum_{j=1}^k \! \sigma^*(v_j^T x)$ where $v_1, \dots, v_k$ are unit …