Finite-Sample Analysis of Decentralized Temporal-Difference Learning with Linear Function Approximation

Jun Sun, Gang Wang, Georgios B. Giannakis, Qinmin Yang, Zaiyue Yang

Type: Preprint

Publication Date: 2019-01-01

Citations: 7

DOI: https://doi.org/10.48550/arxiv.1911.00934

View Publication

Locations

arXiv (Cornell University) - View
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	Exact Formulas for Finite-Time Estimation Errors of Decentralized Temporal Difference Learning with Linear Function Approximation	2022	Xingang Guo Bin Hu
+	Finite-Sample Analyses for Fully Decentralized Multi-Agent Reinforcement Learning.	2018	Kaiqing Zhang Zhuoran Yang Han Liu Tong Zhang Tamer Başar
+	Finite-Time Analysis of Distributed TD(0) with Linear Function Approximation for Multi-Agent Reinforcement Learning	2019	Thinh T. Doan Siva Theja Maguluri Justin Romberg
+ PDF Chat	Byzantine-Resilient Decentralized TD Learning with Linear Function Approximation	2021	Zhaoxian Wu Han Shen Tianyi Chen Qing Ling
+	Finite-Time Performance of Distributed Temporal Difference Learning with Linear Function Approximation	2019	Thinh T. Doan Siva Theja Maguluri Justin Romberg
+	Finite-Time Performance of Distributed Temporal Difference Learning with Linear Function Approximation	2019	Thinh T. Doan Siva Theja Maguluri Justin Romberg
+ PDF Chat	Finite-Sample Analysis for Decentralized Batch Multiagent Reinforcement Learning With Networked Agents	2021	Kaiqing Zhang Zhuoran Yang Han Liu Tong Zhang Tamer Başar
+	Finite-Sample Analysis For Decentralized Batch Multi-Agent Reinforcement Learning With Networked Agents	2018	Kaiqing Zhang Zhuoran Yang Han Liu Tong Zhang Tamer Başar
+	Control Theoretic Analysis of Temporal Difference Learning	2021	Donghwan Lee
+ PDF Chat	Analysis of Off-Policy Multi-Step TD-Learning with Linear Function Approximation	2024	Donghwan Lee
+	Finite-Time Convergence Rates of Decentralized Stochastic Approximation with Applications in Multi-Agent and Multi-Task Learning	2020	Sihan Zeng Thinh T. Doan Justin Romberg
+ PDF Chat	Finite-Time Convergence Rates of Decentralized Stochastic Approximation With Applications in Multi-Agent and Multi-Task Learning	2022	Sihan Zeng Thinh T. Doan Justin Romberg
+ PDF Chat	Sample and Communication Efficient Fully Decentralized MARL Policy Evaluation via a New Approach: Local TD update	2024	Fnu Hairi Zifan Zhang Jia Liu
+ PDF Chat	Byzantine-Resilient Decentralized Policy Evaluation With Linear Function Approximation	2021	Zhaoxian Wu Han Shen Tianyi Chen Qing Ling
+	Fast Multi-Agent Temporal-Difference Learning via Homotopy Stochastic Primal-Dual Optimization	2019	Dong-Sheng Ding Xiaohan Wei Zhuoran Yang Zhaoran Wang Mihailo R. Jovanović
+	Regret Bounds for Decentralized Learning in Cooperative Multi-Agent Dynamical Systems	2020	Seyed Mohammad Asghari Yi Ouyang Ashutosh Nayyar
+	Regret Analysis of Distributed Online LQR Control for Unknown LTI Systems	2021	T. T. Chang Shahin Shahrampour
+	Finite-Time Analysis of Fully Decentralized Single-Timescale Actor-Critic	2022	Qijun Luo Xiao Li
+	Finite-Time Analysis of Temporal Difference Learning: Discrete-Time Linear System Perspective	2022	Donghwan Lee Do Wan Kim
+	Proximal Gradient Temporal Difference Learning: Stable Reinforcement Learning with Polynomial Sample Complexity	2020	Bo Liu Ian Gemp Mohammad Ghavamzadeh Ji Liu Sridhar Mahadevan Marek Petrik

Works That Cite This (6)

Action	Title	Year	Authors
+ PDF Chat	On the Convergence of Consensus Algorithms with Markovian noise and Gradient Bias	2020	Hoi-To Wai
+	On the Convergence of Consensus Algorithms with Markovian Noise and Gradient Bias	2020	Hoi-To Wai
+ PDF Chat	Online Robust Reinforcement Learning with Model Uncertainty	2021	Yue Wang Shaofeng Zou
+ PDF Chat	Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximation	2021	Yue Wang Shaofeng Zou Yi Zhou
+ PDF Chat	Byzantine-Resilient Decentralized Policy Evaluation With Linear Function Approximation	2021	Zhaoxian Wu Han Shen Tianyi Chen Qing Ling
+ PDF Chat	Distributed TD(0) With Almost No Communication	2023	Rui Liu Alex Olshevsky

Works Cited by This (18)

Action	Title	Year	Authors
+	Finite-Sample Analysis of Proximal Gradient TD Algorithms	2020	Bo Liu Ji Liu Mohammad Ghavamzadeh Sridhar Mahadevan Marek Petrik
+	Learning to Communicate with Deep Multi-Agent Reinforcement Learning	2016	Jakob Foerster Yannis Assael Nando de Freitas Shimon Whiteson
+	Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents	2018	Kaiqing Zhang Zhuoran Yang Han Liu Tong Zhang Tamer Başar
+	Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization	2019	Gang Wang Georgios B. Giannakis Jie Chen
+	Lyapunov Theory for Discrete Time Systems	2018	Nicoletta Bof Ruggero Carli Luca Schenato
+	Tight Linear Convergence Rate of ADMM for Decentralized Optimization	2019	Meng Ma Bingcong Li Georgios B. Giannakis
+	Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization	2018	Angelia Nedić Alex Olshevsky Michael Rabbat
+	Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization	2018	Hoi-To Wai Zhuoran Yang Zhaoran Wang Mingyi Hong
+ PDF Chat	Finite Sample Analyses for TD(0) With Function Approximation	2018	Gal Dalal Balázs Szörényi Gugan Thoppe Shie Mannor
+ PDF Chat	Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview	2019	Yuejie Chi Yue M. Lu Yuxin Chen