Regularization Matters in Policy Optimization

Zhuang Liu, Xuanlin Li, Bingyi Kang, Trevor Darrell

Type: Preprint

Publication Date: 2019-01-01

Citations: 19

DOI: https://doi.org/10.48550/arxiv.1910.09191

View Publication

Locations

arXiv (Cornell University) - View - PDF
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	Regularization Matters in Policy Optimization -- An Empirical Study on Continuous Control	2019	Zhuang Liu Xuanlin Li Bingyi Kang Trevor Darrell
+	Control Regularization for Reduced Variance Reinforcement Learning	2019	Richard Cheng Abhinav Verma Gábor Orosz Swarat Chaudhuri Yisong Yue Joel W. Burdick
+	The Definitive Guide to Policy Gradients in Deep Reinforcement Learning: Theory, Algorithms and Implementations	2024	Matthias Lehmann
+	Entropy Regularized Reinforcement Learning with Cascading Networks	2022	Riccardo Della Vecchia Alena Shilova Pierre‐Marie Preux Riad Akrour
+ PDF Chat	Learning Value Functions in Deep Policy Gradients using Residual Variance	2021	Yannis Flet-Berliac Reda Ouhamma Odalric-Ambrym Maillard Pierre‐Marie Preux
+	Learning Value Functions in Deep Policy Gradients using Residual Variance	2020	Yannis Flet-Berliac Reda Ouhamma Odalric-Ambrym Maillard Pierre‐Marie Preux
+	Robust Policy Optimization in Deep Reinforcement Learning	2022	Md. Masudur Rahman Yexiang Xue
+	Sample Dropout: A Simple yet Effective Variance Reduction Technique in Deep Policy Optimization	2023	Zichuan Lin Xiapeng Wu Mingfei Sun Deheng Ye Qiang Fu Wei Yang Wei Liu
+	Where Did My Optimum Go?: An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methods	2018	Peter Henderson Joshua Romoff Joëlle Pineau
+ PDF Chat	Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergence	2023	Wenhao Zhan Shicong Cen Baihe Huang Yuxin Chen Jason D. Lee Yuejie Chi
+	Learning to Optimize for Reinforcement Learning	2023	Qingfeng Lan A. Rupam Mahmood Shuicheng Yan Zhongwen Xu
+	How to Make Deep RL Work in Practice	2020	Nirnai Rao Elie Aljalbout Axel Sauer Sami Haddadin
+	Revisiting Estimation Bias in Policy Gradients for Deep Reinforcement Learning	2023	Haoxuan Pan Deheng Ye Xiaoming Duan Qiang Fu Wei Yang Jianping He Mingfei Sun
+	NADPEx: An on-policy temporally consistent exploration method for deep reinforcement learning	2018	Sirui Xie Junning Huang Lanxin Lei Chunxiao Liu Zheng Ma Wei Zhang Liang Lin
+	NADPEx: An on-policy temporally consistent exploration method for deep reinforcement learning	2018	Sirui Xie Junning Huang Lanxin Lei Chunxiao Liu Zheng Ma Wei Zhang Liang Lin
+ PDF Chat	Improving Deep Reinforcement Learning by Reducing the Chain Effect of Value and Policy Churn	2024	Hongyao Tang Glen Berseth
+	Cautious Policy Programming: Exploiting KL Regularization in Monotonic Policy Improvement for Reinforcement Learning	2021	Lingwei Zhu Toshinori Kitamura Takamitsu Matsubara
+ PDF Chat	Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning	2024	Michal Nauman Michał Bortkiewicz Mateusz Ostaszewski Piotr Miłoś T. P. Trzcinski Marek Cygan
+	On Multi-objective Policy Optimization as a Tool for Reinforcement Learning.	2021	Abbas Abdolmaleki Sandy H. Huang Giulia Vezzani Bobak Shahriari Jost Tobias Springenberg Shruti Mishra Dhruva Tb Arunkumar Byravan Konstantinos Bousmalis András György
+	Handling Cost and Constraints with Off-Policy Deep Reinforcement Learning	2023	J. Markowitz Jesse L. Silverberg Gary S. Collins

Works That Cite This (16)

Action	Title	Year	Authors
+	Training Larger Networks for Deep Reinforcement Learning	2021	Kei Ota Devesh K. Jha Asako Kanezaki
+	Proximal Policy Gradient: PPO with Policy Gradient	2020	Ju-Seung Byun Byung Moon Kim Huamin Wang
+	A Hybrid Stochastic Policy Gradient Algorithm for Reinforcement Learning	2020	Nhan H. Pham Lam M. Nguyen Dzung T. Phan Phuong Ha Nguyen Marten van Dijk Quoc Tran-Dinh
+	Regularizing Action Policies for Smooth Control with Reinforcement Learning	2020	Siddharth Mysore Bassel Mabsout Renato Mancuso Kate Saenko
+	Deep learning in electron microscopy	2020	Jeffrey M. Ede
+ PDF Chat	Regularizing Action Policies for Smooth Control with Reinforcement Learning	2021	Siddharth Mysore Bassel Mabsout Renato Mancuso Kate Saenko
+ PDF Chat	Learning from Simulation, Racing in Reality	2021	Eugenio Chisari Alexander Liniger Alisa Rupenyan Luc Van Gool John Lygeros
+ PDF Chat	IPAPRec: A Promising Tool for Learning High-Performance Mapless Navigation Skills With Deep Reinforcement Learning	2022	Wei Zhang Yunfeng Zhang Ning Liu Kai Ren Pengfei Wang
+	F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning	2020	Wenhao Li Bo Jin Xiangfeng Wang Junchi Yan Hongyuan Zha
+	Softmax with Regularization: Better Value Estimation in Multi-Agent Reinforcement Learning.	2021	Ling Pan Tabish Rashid Bei Peng Longbo Huang Shimon Whiteson

Works Cited by This (33)

Action	Title	Year	Authors
+ PDF Chat	Deep Residual Learning for Image Recognition	2016	Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun
+	RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning	2016	Yan Duan John Schulman Xi Chen Peter L. Bartlett Ilya Sutskever Pieter Abbeel
+	Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks	2017	Chelsea Finn Pieter Abbeel Sergey Levine
+ PDF Chat	Domain randomization for transferring deep neural networks from simulation to the real world	2017	Josh Tobin Rachel Fong Alex Ray Jonas Schneider Wojciech Zaremba Pieter Abbeel
+	A unified view of entropy-regularized Markov decision processes	2017	Gergely Neu Anders Jönsson Vicenç Gómez
+	Proximal Policy Optimization Algorithms	2017	John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov
+	Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments	2017	Maruan Al-Shedivat Trapit Bansal Yuri Burda Ilya Sutskever Igor Mordatch Pieter Abbeel
+	IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures	2018	Lasse Espeholt Hubert Soyer Rémi Munos Karen Simonyan Volodymir Mnih Tom Ward Yotam Doron Vlad Firoiu Tim Harley Iain Dunning
+	Addressing Function Approximation Error in Actor-Critic Methods	2018	Scott Fujimoto Herke van Hoof David Meger
+	Meta-Reinforcement Learning of Structured Exploration Strategies	2018	Abhishek Gupta Russell Mendonca YuXuan Liu Pieter Abbeel Sergey Levine