An Improved Analysis of Gradient Tracking for Decentralized Machine Learning

Anastasia Koloskova, Tao Lin, Sebastian U. Stich

Type: Preprint

Publication Date: 2022-01-01

Citations: 24

DOI: https://doi.org/10.48550/arxiv.2202.03836

View Publication

Locations

arXiv (Cornell University) - View - PDF
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	Decentralized Gradient Tracking with Local Steps	2023	Yue Liu Tao Lin Anastasia Koloskova Sebastian U. Stich
+ PDF Chat	Decentralized gradient tracking with local steps	2024	Yue Liu Tao Lin Anastasia Koloskova Sebastian U. Stich
+ PDF Chat	The Effectiveness of Local Updates for Decentralized Learning under Data Heterogeneity	2024	Tongle Wu Ying Sun
+	A Flexible Gradient Tracking Algorithmic Framework for Decentralized Optimization	2023	Albert S. Berahas Raghu Bollapragada Shagun Gupta
+	Balancing Communication and Computation in Gradient Tracking Algorithms for Decentralized Optimization	2023	Albert S. Berahas Raghu Bollapragada Shagun Gupta
+ PDF Chat	Stability and Generalization of Decentralized Stochastic Gradient Descent	2021	Tao Sun Dongsheng Li Bao Wang
+	Stability and Generalization of the Decentralized Stochastic Gradient Descent	2021	Tao Sun Dongsheng Li Bao Wang
+ PDF Chat	Enhancing Convergence of Decentralized Gradient Tracking under the KL Property	2024	Xiaokai Chen Tianyu Cao Gesualdo Scutari
+	On the Performance of Gradient Tracking with Local Updates	2022	Edward Duc Hien Nguyen Sulaiman A. Alghunaim Kun Yuan César A. Uribe
+	An Improved Convergence Analysis for Decentralized Online Stochastic Non-Convex Optimization	2021	Ran Xin Usman A. Khan Soummya Kar
+ PDF Chat	On the Performance of Gradient Tracking with Local Updates	2023	Edward Duc Hien Nguyen Sulaiman A. Alghunaim Kun Yuan César A. Uribe
+	Improving the Sample and Communication Complexity for Decentralized Non-Convex Optimization: A Joint Gradient Estimation and Tracking Approach	2019	Haoran Sun Songtao Lu Mingyi Hong
+	Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent	2020	Dominic Richards Patrick Rebeschini
+	Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent	2018	Dominic Richards Patrick Rebeschini
+ PDF Chat	Tighter Analysis for Decentralized Stochastic Gradient Method: Impact of Data Homogeneity	2024	Qiang Li Hoi-To Wai
+	An Enhanced Gradient-Tracking Bound for Distributed Online Stochastic Convex Optimization	2023	Sulaiman A. Alghunaim Kun Yuan
+	A fast randomized incremental gradient method for decentralized non-convex optimization	2020	Ran Xin Usman A. Khan Soummya Kar
+	A Stochastic Gradient Tracking Algorithm for Decentralized Optimization With Inexact Communication	2023	Suhail M. Shah Raghu Bollapragada
+	Convergence Analysis of Decentralized ASGD	2023	Mauro DL Tosi Martin Theobald
+	Optimized Gradient Tracking for Decentralized Online Learning	2023	Shivangi Dubey Sharma Ketan Rajawat

Works That Cite This (17)

Action	Title	Year	Authors
+	Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Data	2022	Batiste Le Bars Aurélien Bellet Marc Tommasi Erick Lavoie Anne-Marie Kermarrec
+ PDF Chat	Cross-feature Contrastive Loss for Decentralized Deep Learning on Heterogeneous Data	2024	Sai Aparna Aketi Kaushik Roy
+ PDF Chat	Recent Theoretical Advances in Decentralized Distributed Convex Optimization	2022	Eduard Gorbunov Alexander Rogozin Aleksandr Beznosikov Darina Dvinskikh Alexander Gasnikov
+ PDF Chat	A Unified Momentum-based Paradigm of Decentralized SGD for Non-Convex Models and Heterogeneous Data	2024	Haizhou Du Chaoqian Cheng Chengdong Ni
+ PDF Chat	Decentralized Asynchronous Nonconvex Stochastic Optimization on Directed Graphs	2023	Vyacheslav Kungurtsev Mahdi Morafah Tara Javidi Gesualdo Scutari
+ PDF Chat	An enhanced gradient-tracking bound for distributed online stochastic convex optimization	2023	Sulaiman A. Alghunaim Kun Yuan
+	DADAO: Decoupled Accelerated Decentralized Asynchronous Optimization	2022	Adel Nabli Edouard Oyallon
+	Connectivity-Aware Semi-Decentralized Federated Learning over Time-Varying D2D Networks	2023	Rohit Parasnis Seyyedali Hosseinalipour Yun-Wei Chu Mung Chiang Christopher G. Brinton
+ PDF Chat	Taming Subnet-Drift in D2D-Enabled Fog Learning: A Hierarchical Gradient Tracking Approach	2024	Evan Chen Shiqiang Wang Christopher G. Brinton
+ PDF Chat	Jointly Improving the Sample and Communication Complexities in Decentralized Stochastic Minimax Optimization	2024	Xuan Zhang Gabriel Mancino-Ball Necdet Serhat Aybat Yangyang Xu

Works Cited by This (0)

Action	Title	Year	Authors