A Dynamical Systems Perspective on Nesterov Acceleration

Michael Muehlebach, Michael I. Jordan

Type: Preprint

Publication Date: 2019-01-01

Citations: 38

DOI: https://doi.org/10.48550/arxiv.1905.07436

View Publication

Locations

arXiv (Cornell University) - View
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	A Continuized View on Nesterov Acceleration	2021	Raphaël Berthier Francis R. Bach Nicolas Flammarion Pierre Gaillard Adrien Taylor
+	A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights	2015	Weijie Su Stephen Boyd Emmanuel J. Candès
+	A differential equation for modeling Nesterov's accelerated gradient method: theory and insights	2016	Weijie Su Stephen Boyd Emmanuel J. Candès
+	A differential equation for modeling Nesterov's accelerated gradient method	2016	SuWeijie BoydStephen J CandèsEmmanuel
+	Continuous-Time Analysis of Accelerated Gradient Methods via Conservation Laws in Dilated Coordinate Systems	2022	Jaewook J. Suh Gyumin Roh Ernest K. Ryu
+	A Differential Equation for Modeling Nesterovâ€™s Accelerated Gradient Method: Theory and Insights	2014	Weijie Su Stephen Boyd Emmanuel J. Candès
+	A Continuous-time Perspective for Modeling Acceleration in Riemannian Optimization	2019	Foivos Alimisis Antonio Orvieto Gary Bécigneul Aurélien Lucchi
+	A Continuous-time Perspective for Modeling Acceleration in Riemannian Optimization.	2020	Foivos Alimisis Antonio Orvieto Gary Bécigneul Aurélien Lucchi
+	Accelerated First-Order Methods: Differential Equations and Lyapunov Functions	2019	Jonathan W. Siegel
+	Accelerated First-Order Methods: Differential Equations and Lyapunov Functions	2019	Jonathan W. Siegel
+	Dissipativity Theory for Nesterov's Accelerated Method	2017	Bin Hu Laurent Lessard
+	Dissipativity Theory for Nesterov's Accelerated Method	2017	Bin Hu Laurent Lessard
+	An explicit time-integrator for non-smooth dynamics	2022	Jean Di Stasio David Dureisseix Anthony Gravouil Gabriel Georges Thomas Homolle
+	Understanding the Acceleration Phenomenon via High-Resolution Differential Equations	2018	Bin Shi Simon S. Du Michael I. Jordan Weijie Su
+	Nesterov's method with decreasing learning rate leads to accelerated stochastic gradient descent	2019	Maxime Laborde Adam M. Oberman
+	Gradient Norm Minimization of Nesterov Acceleration: $o(1/k^3)$	2022	Shuo Chen Bin Shi Ya-xiang Yuan
+	A High-Order Implicit Time Integration Method for Linear and Nonlinear Dynamics with Efficient Computation of Accelerations	2024	Daniel J. Shea Xiaoran Zhang Shayan Mohammadian Chongmin Song
+	A revisit on Nesterov acceleration for linear ill-posed problems	2024	Duo Liu Qin Huang Qinian Jin
+ PDF Chat	Understanding the acceleration phenomenon via high-resolution differential equations	2021	Bin Shi Simon S. Du Michael I. Jordan Weijie Su
+	On Symplectic Optimization	2018	Michael Betancourt Michael I. Jordan Ashia C. Wilson

Works That Cite This (36)

Action	Title	Year	Authors
+	Symplectic Accelerated Optimization on SO(3) with Lie Group Variational Integrators	2020	Harsh Sharma Taeyoung Lee Mayuresh Patil Craig A. Woolsey
+	On the asymptotic stability of proximal algorithms for convex optimization problems with multiple non-smooth regularizers	2022	Ibrahim K. Ozaslan Sepideh Hassan-Moghaddam Mihailo R. Jovanović
+	Design of generalized fractional order gradient descent method	2018	Yiheng Wei Yü Kang Weidi Yin Yong Wang
+	Gradient Flows and Accelerated Proximal Splitting Methods.	2019	Guilherme França Daniel P. Robinson René Vidal
+ PDF Chat	An Extension of the Second Order Dynamical System that Models Nesterov’s Convex Gradient Method	2020	Cristian Daniel Alecsa Szilárd Csaba László Titus Pinţa
+ PDF Chat	Implicit Runge-Kutta Methods for Accelerated Unconstrained Convex Optimization	2020	Ruijuan Chen Xiuting Li
+ PDF Chat	On dissipative symplectic integration with applications to gradient-based optimization	2021	Guilherme França Michael I. Jordan René Vidal
+	Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization	2021	Peiyuan Zhang Antonio Orvieto Hadi Daneshmand Thomas Hofmann Roy S. Smith
+	The long time behavior and the rate of convergence of symplectic convex algorithms obtained via splitting discretizations of inertial damping systems	2020	Cristian Daniel Alecsa
+ PDF Chat	Proximal gradient flow and Douglas–Rachford splitting dynamics: Global exponential stability via integral quadratic constraints	2020	Sepideh Hassan-Moghaddam Mihailo R. Jovanović

Works Cited by This (11)

Action	Title	Year	Authors
+	Some methods of speeding up the convergence of iteration methods	1964	B. T. Polyak
+	Principles of mathematical analysis	1964	Walter Rudin
+	A geometric alternative to Nesterov's accelerated gradient descent	2015	Sébastien Bubeck Yin Tat Lee Mohit Singh
+	Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity	2016	Hédy Attouch Zaki Chbani Juan Peypouquet Patrick Redont
+	A differential equation for modeling Nesterov's accelerated gradient method: theory and insights	2016	Weijie Su Stephen Boyd Emmanuel J. Candès
+	Integration Methods and Optimization Algorithms	2017	Damien Scieur Vincent Roulet Francis Bach Alexandre d’Aspremont
+ PDF Chat	The Approximate Duality Gap Technique: A Unified Theory of First-Order Methods	2019	Jelena Diakonikolas Lorenzo Orecchia
+	Geometric Numerical Integration	2002	Ernst Hairer Gerhard Wanner Christian Lubich
+ PDF Chat	A variational perspective on accelerated methods in optimization	2016	Andre Wibisono Ashia C. Wilson Michael I. Jordan
+ PDF Chat	Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints	2016	Laurent Lessard Benjamin Recht Andrew Packard