On the Powerball Method: Variants of Descent Methods for Accelerated Optimization
On the Powerball Method: Variants of Descent Methods for Accelerated Optimization
We propose a new method to accelerate the convergence of optimization algorithms. This method, termed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Powerball method</i> , simply adds a power coefficient <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\gamma \in [0{,}1$ </tex-math></inline-formula> ) to the gradient used in various gradient-based optimization schemes. In its essence, the Powerball method can …