Analysis of Off-Policy Multi-Step TD-Learning with Linear Function
Approximation
Analysis of Off-Policy Multi-Step TD-Learning with Linear Function
Approximation
This paper analyzes multi-step TD-learning algorithms within the `deadly triad' scenario, characterized by linear function approximation, off-policy learning, and bootstrapping. In particular, we prove that n-step TD-learning algorithms converge to a solution as the sampling horizon n increases sufficiently. The paper is divided into two parts. In the first part, …