A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning

Sihan Zeng, Thinh T. Doan, Justin Romberg

Type: Preprint

Publication Date: 2021-01-01

Citations: 3

DOI: https://doi.org/10.48550/arxiv.2109.14756

View Publication

Locations

arXiv (Cornell University) - View - PDF
DataCite API - View

Similar Works

Action	Title	Year	Authors
+ PDF Chat	A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning	2024	Sihan Zeng Thinh T. Doan Justin Romberg
+ PDF Chat	Fast Two-Time-Scale Stochastic Gradient Method with Applications in Reinforcement Learning	2024	Sihan Zeng Thinh T. Doan
+	A model-free first-order method for linear quadratic regulator with $\tilde{O}(1/\varepsilon)$ sampling complexity	2022	Caleb Ju Georgios Kotsalis Guanghui Lan
+	Global Convergence of Two-timescale Actor-Critic for Solving Linear Quadratic Regulator	2022	Xuyang Chen Jingliang Duan Yingbin Liang Lin Zhao
+	A Finite Time Analysis of Two Time-Scale Actor Critic Methods	2020	Yue Wu Weitong Zhang Pan Xu Quanquan Gu
+	A Finite Time Analysis of Two Time-Scale Actor Critic Methods	2020	Yue Wu Weitong Zhang Pan Xu Quanquan Gu
+ PDF Chat	Global Convergence of Two-Timescale Actor-Critic for Solving Linear Quadratic Regulator	2023	Xuyang Chen Jingliang Duan Yingbin Liang Lin Zhao
+ PDF Chat	Sample Complexity of the Linear Quadratic Regulator: A Reinforcement Learning Lens	2024	Amirreza Neshaei Moghaddam Alex Olshevsky Bahman Gharesifard
+	Finite-time analysis of single-timescale actor-critic	2022	Xuyang Chen Lin Zhao
+	On the Sample Complexity of Actor-Critic Method for Reinforcement Learning with Function Approximation	2019	Harshat Kumar Alec Koppel Alejandro Ribeiro
+	Tight Finite Time Bounds of Two-Time-Scale Linear Stochastic Approximation with Markovian Noise	2024	Shaan ul Haque Sajad Khodadadian Siva Theja Maguluri
+	Non-asymptotic Convergence Analysis of Two Time-scale (Natural) Actor-Critic Algorithms	2020	Tengyu Xu Zhe Wang Yingbin Liang
+ PDF Chat	Sublinear Regret for a Class of Continuous-Time Linear--Quadratic Reinforcement Learning Problems	2024	Yilie Huang Yanwei Jia Xun Zhou
+	A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic	2020	Mingyi Hong Hoi To Wai Zhaoran Wang Zhuoran Yang
+ PDF Chat	Accelerated Multi-Time-Scale Stochastic Approximation: Optimal Complexity and Applications in Reinforcement Learning and Multi-Agent Games	2024	Sihan Zeng Thinh T. Doan
+	Finite-Time Complexity of Online Primal-Dual Natural Actor-Critic Algorithm for Constrained Markov Decision Processes	2021	Sihan Zeng Thinh T. Doan Justin Romberg
+	Finite Sample Analysis of Two-Time-Scale Natural Actor-Critic Algorithm	2021	Sajad Khodadadian Thinh T. Doan Justin Romberg Siva Theja Maguluri
+	Finite-Time Error Bounds for Greedy-GQ	2022	Yue Wang Yi Zhou Shaofeng Zou
+	On the Global Convergence of Actor-Critic: A Case for Linear Quadratic Regulator with Ergodic Cost	2019	Zhuoran Yang Yongxin Chen Mingyi Hong Zhaoran Wang
+	Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizon	2020	Matteo Basei Xin Guo Anran Hu Yufei Zhang

Works That Cite This (1)

Action	Title	Year	Authors
+ PDF Chat	Finite-Time Complexity of Online Primal-Dual Natural Actor-Critic Algorithm for Constrained Markov Decision Processes	2022	Sihan Zeng Thinh T. Doan Justin Romberg

Works Cited by This (18)

Action	Title	Year	Authors
+ PDF Chat	Convergence rate of linear two-time-scale stochastic approximation	2004	Vijay R. Konda John N. Tsitsiklis
+	Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator	2018	Maryam Fazel Rong Ge Sham M. Kakade Mehran Mesbahi
+	On Markov Chain Gradient Descent	2018	Tao Sun Yuejiao Sun Wotao Yin
+	Finite-Time Performance Bounds and Adaptive Learning Rate Selection for Two Time-Scale Reinforcement Learning	2019	Harsh Gupta R. Srikant Lei Ying
+ PDF Chat	A Tale of Two-Timescale Reinforcement Learning with the Tightest Finite-Time Bound	2020	Gal Dalal Balázs Szörényi Gugan Thoppe
+	Convergence Rates of Accelerated Markov Gradient Descent with Applications in Reinforcement Learning	2020	Thinh T. Doan Lam M. Nguyen Nhan H. Pham Justin Romberg
+	Toward a theory of optimization for over-parameterized systems of non-linear equations: the lessons of deep learning.	2020	Chaoyue Liu Libin Zhu Mikhail A. Belkin
+	Finite-Time Analysis of Stochastic Gradient Descent under Markov Randomness	2020	Thinh T. Doan Lam M. Nguyen Nhan H. Pham Justin Romberg
+	Non-asymptotic Convergence Analysis of Two Time-scale (Natural) Actor-Critic Algorithms	2020	Tengyu Xu Zhe Wang Yingbin Liang
+	A Finite Time Analysis of Two Time-Scale Actor Critic Methods	2020	Yue Wu Weitong Zhang Pan Xu Quanquan Gu