On Instrumental Variable Regression for Deep Offline Policy Evaluation

Yutian Chen, Liyuan Xu, Çaǧlar Gülçehre, Tom Le Paine, Arthur Gretton, Nando de Freitas, Arnaud Doucet

Type: Preprint

Publication Date: 2021-01-01

Citations: 2

DOI: https://doi.org/10.48550/arxiv.2105.10148

View Publication

Locations

arXiv (Cornell University) - View - PDF
DataCite API - View

Similar Works

Action	Title	Year	Authors
+	Conservative Q-Learning for Offline Reinforcement Learning	2020	Aviral Kumar Aurick Zhou George Tucker Sergey Levine
+	Conservative Q-Learning for Offline Reinforcement Learning	2020	Aviral Kumar Aurick Zhou George Tucker Sergey Levine
+	Q-Value Weighted Regression: Reinforcement Learning with Limited Data	2021	Piotr Kozakowski Łukasz Kaiser Henryk Michalewski Afroz Mohiuddin Katarzyna Kańska
+	IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies	2023	Philippe Hansen-Estruch Ilya Kostrikov Michael Jänner Jakub Grudzien Kuba Sergey Levine
+ PDF Chat	Q-Value Weighted Regression: Reinforcement Learning with Limited Data	2022	Piotr Kozakowski Łukasz Kaiser Henryk Michalewski Afroz Mohiuddin Katarzyna Kańska
+	On Finite-Sample Analysis of Offline Reinforcement Learning with Deep ReLU Networks.	2021	Thanh Nguyen-Tang Sunil Gupta Hung Tran-The Svetha Venkatesh
+	Instrumental Variable Value Iteration for Causal Offline Reinforcement Learning	2021	Luofeng Liao Zuyue Fu Zhuoran Yang Zhaoran Wang Mladen Kolar Zhaoran Wang
+ PDF Chat	Orthogonalized Estimation of Difference of $Q$-functions	2024	Angela Zhou
+	Revisiting Bellman Errors for Offline Model Selection	2023	Joshua P. Zitovsky Daniel de Marchi Rishabh Agarwal Michael R. Kosorok
+	Offline Reinforcement Learning with Implicit Q-Learning	2021	Ilya Kostrikov Ashvin Nair Sergey Levine
+	Confidence-Conditioned Value Functions for Offline Reinforcement Learning	2022	Joey Hong Aviral Kumar Sergey Levine
+ PDF Chat	Is Value Learning Really the Main Bottleneck in Offline RL?	2024	Seohong Park Kevin Frans Sergey Levine Aviral Kumar
+ PDF Chat	POPO: Pessimistic Offline Policy Optimization	2022	Qiang He Xinwen Hou Yu Liu
+ PDF Chat	Learning Decision Policies with Instrumental Variables through Double Machine Learning	2024	Daqian Shao Ashkan Soleymani Francesco Quinzan Marta Kwiatkowska
+	Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning	2017	Shixiang Gu Timothy Lillicrap Zoubin Ghahramani Richard E. Turner Bernhard Schölkopf Sergey Levine
+	Learning Bellman Complete Representations for Offline Policy Evaluation	2022	Jonathan Chang Kaiwen Wang Nathan Kallus Wen Sun
+	An Instrumental Variable Approach to Confounded Off-Policy Evaluation	2022	Xu Yang Zhu Jin Chengchun Shi Shikai Luo Rui Song
+	On Multi-objective Policy Optimization as a Tool for Reinforcement Learning.	2021	Abbas Abdolmaleki Sandy H. Huang Giulia Vezzani Bobak Shahriari Jost Tobias Springenberg Shruti Mishra Dhruva Tb Arunkumar Byravan Konstantinos Bousmalis András György
+ PDF Chat	Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL	2024	Yu Luo Tianying Ji Fuchun Sun Jianwei Zhang Huazhe Xu Xianyuan Zhan
+	Imitation-Regularized Offline Learning	2019	Yifei Ma Yu-Xiang Wang Balakrishnan Narayanaswamy

Works That Cite This (2)

Action	Title	Year	Authors
+ PDF Chat	Adaptive Estimation and Uniform Confidence Bands for Nonparametric Structural Functions and Elasticities	2024	Xiaohong Chen Timothy Christensen Sid Kankanala
+ PDF Chat	Reading and conducting instrumental variable studies: guide, glossary, and checklist	2024	Venexia Walker Eleanor Sanderson Michael G. Levin Scott M Damraurer Timothy Feeney Neil M Davies

Works Cited by This (37)

Action	Title	Year	Authors
+ PDF Chat	Chapter 77 Linear Inverse Problems in Structural Econometrics Estimation Based on Spectral Decomposition and Regularization	2007	Marine Carrasco Jean‐Pierre Florens Èric Renault
+	Doubly Robust Policy Evaluation and Learning	2011	Miroslav Dudík John Langford Lihong Li
+	Instrumental Variable Estimation of Nonparametric Models	2003	Whitney K. Newey James L. Powell
+ PDF Chat	Learning near-optimal policies with Bellman-residual minimization based fitted policy iteration and a single sample path	2007	András Antos Csaba Szepesvári Rémi Munos
+	Finite-Time Bounds for Fitted Value Iteration	2008	Rémi Munos Csaba Szepesvári
+ PDF Chat	Identification of Causal Effects Using Instrumental Variables	1996	Joshua D. Angrist Guido W. Imbens Donald B. Rubin
+	Continuous control with deep reinforcement learning	2015	Timothy Lillicrap Jonathan J. Hunt Alexander Pritzel Nicolas Heess Tom Erez Yuval Tassa David Silver Daan Wierstra
+	Deep IV: a flexible approach for counterfactual prediction	2017	Jason Hartford Greg Lewis Kevin Leyton‐Brown Matt Taddy
+	Adversarial Generalized Method of Moments	2018	Greg Lewis Vasilis Syrgkanis
+	Optimal sup-norm rates and uniform inference on nonlinear functionals of nonparametric IV regression	2018	Xiaohong Chen Timothy Christensen