Is Pessimism Provably Efficient for Offline RL

Ying Jin, Zhuoran Yang, Zhaoran Wang

Type: Preprint

Publication Date: 2020-12-30

Citations: 23

Locations

arXiv (Cornell University) - View

Similar Works

Action	Title	Year	Authors
+	Is Pessimism Provably Efficient for Offline RL?	2020	Ying Jin Zhuoran Yang Zhaoran Wang
+	Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexity	2022	Laixi Shi Gen Li Yuting Wei Yuxin Chen Yuejie Chi
+	VIPeR: Provably Efficient Algorithm for Offline RL with Neural Function Approximation	2023	Thanh Nguyen-Tang Raman Arora
+	Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes	2022	Miao Lu Yifei Min Zhaoran Wang Zhuoran Yang
+	Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism	2022	Ming Yin Yaqi Duan Mengdi Wang Yuxiang Wang
+	Towards Instance-Optimal Offline Reinforcement Learning with Pessimism	2021	Minghao Yin Yu-Xiang Wang
+ PDF Chat	Optimistic Model Rollouts for Pessimistic Offline Policy Optimization	2024	Yuanzhao Zhai Yiying Li Zijian Gao Xudong Gong Kele Xu Dawei Feng Bo Ding Huaimin Wang
+	Optimistic Model Rollouts for Pessimistic Offline Policy Optimization	2024	Yuanzhao Zhai Yiying Li Zijian Gao Xudong Gong Kele Xu Dawei Feng Bo Ding Huaimin Wang
+ PDF Chat	Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism	2022	Paria Rashidinejad Banghua Zhu Cong Ma Jiantao Jiao Stuart Russell
+	Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Belief	2022	Kaiyang Guo Yunfeng Shao Yanhui Geng
+	Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism	2021	Paria Rashidinejad Banghua Zhu Cong Ma Jiantao Jiao Stuart Russell
+	Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learning	2022	Chenjia Bai Lingxiao Wang Zhuoran Yang Zhihong Deng Animesh Garg Peng Liu Zhaoran Wang
+	Bi-Level Offline Policy Optimization with Limited Exploration	2023	Wenzhuo Zhou
+ PDF Chat	Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning	2024	Dake Zhang Boxiang Lyu Shuang Qiu Mladen Kolar Tong Zhang
+	Nearly Minimax Optimal Offline Reinforcement Learning with Linear Function Approximation: Single-Agent MDP and Markov Game	2022	Wei Xiong Han Zhong Chengshuai Shi Cong Shen Liwei Wang Tong Zhang
+ PDF Chat	Bellman-consistent Pessimism for Offline Reinforcement Learning	2021	Tengyang Xie Ching-An Cheng Nan Jiang Paul Mineiro Alekh Agarwal
+	Bellman-consistent Pessimism for Offline Reinforcement Learning	2021	Tengyang Xie Ching-An Cheng Nan Jiang Paul Mineiro Alekh Agarwal
+	Towards Tractable Optimism in Model-Based Reinforcement Learning	2020	Aldo Pacchiano Philip Ball Jack Parker-Holder Krzysztof Choromański Stephen Roberts
+ PDF Chat	Bayesian Design Principles for Offline-to-Online Reinforcement Learning	2024	Hao Hu Yiqin Yang Jianing Ye Chengjie Wu Ziqing Mai Yujing Hu Tangjie Lv Changjie Fan Qianchuan Zhao Chongjie Zhang
+	Achieving the Minimax Optimal Sample Complexity of Offline Reinforcement Learning: A DRO-Based Approach	2023	Yue Wang Yuting Hu Jinjun Xiong Shaofeng Zou

Works That Cite This (22)

Action	Title	Year	Authors
+	Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning	2021	Tengyang Xie Nan Jiang Huan Wang Caiming Xiong Yu Bai
+	On Finite-Sample Analysis of Offline Reinforcement Learning with Deep ReLU Networks.	2021	Thanh Nguyen-Tang Sunil Gupta Hung Tran-The Svetha Venkatesh
+	Sample Complexity of Offline Reinforcement Learning with Deep ReLU Networks	2021	Thanh Nguyen-Tang Sunil Gupta Hung Tran-The Svetha Venkatesh
+	Combining Online Learning and Offline Learning for Contextual Bandits with Deficient Support	2021	Hung Tran-The Sunil Gupta Thanh Nguyen-Tang Santu Rana Svetha Venkatesh
+	Heuristic-Guided Reinforcement Learning	2021	Ching-An Cheng Andrey Kolobov Adith Swaminathan
+	Corruption-Robust Offline Reinforcement Learning	2021	Xuezhou Zhang Yiding Chen Junwei Zhu W. Sun
+	Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning	2021	Andrea Zanette Martin J. Wainwright Emma Brunskill
+	Pessimistic Model-based Offline Reinforcement Learning under Partial Coverage	2021	Masatoshi Uehara W. Sun
+	Towards Theoretical Understandings of Robust Markov Decision Processes: Sample Complexity and Asymptotics	2021	Wenhao Yang Liangyu Zhang Zhihua Zhang
+	Representation Learning for Online and Offline RL in Low-rank MDPs	2021	Masatoshi Uehara Xuezhou Zhang W. Sun

Works Cited by This (42)

Action	Title	Year	Authors
+	Assouad, Fano, and Le Cam	1997	Bin Yu
+	An Introduction to Matrix Concentration Inequalities	2015	Joel A. Tropp
+ PDF Chat	The Adaptive Lasso and Its Oracle Properties	2006	Hui Zou
+	Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties	2001	Jianqing Fan Runze Li
+	Model-based Reinforcement Learning and the Eluder Dimension	2014	Ian Osband Benjamin Van Roy
+ PDF Chat	Dynamic Treatment Regimes	2013	Bibhas Chakraborty Susan A. Murphy
+	Ideal spatial adaptation by wavelet shrinkage	1994	David L. Donoho Iain M. Johnstone
+	VIME: Variational Information Maximizing Exploration	2016	Rein Houthooft Xi Chen Yan Duan John Schulman Filip De Turck Pieter Abbeel
+	Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving	2016	Shai Shalev‐Shwartz Shaked Shammah Amnon Shashua
+	Minimax Regret Bounds for Reinforcement Learning	2017	Mohammad Gheshlaghi Azar Ian Osband Rémi Munos