Ergodic theory for controlled Markov chains with stationary inputs
Ergodic theory for controlled Markov chains with stationary inputs
Consider a stochastic process $\boldsymbol{X}$ on a finite state space $\mathsf{X}=\{1,\dots,d\}$. It is conditionally Markov, given a real-valued āinput processā $\boldsymbol{\zeta}$. This is assumed to be small, which is modeled through the scaling, \[\zeta_{t}=\varepsilon\zeta^{1}_{t},\qquad0\le\varepsilon\le1,\] where $\boldsymbol{\zeta}^{1}$ is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions ā¦