A Short Note on Stationary Distributions of Unichain Markov Decision Processes
Ronald Ortner
Abstract
Dealing with unichain MDPs, we consider stationary distributions of policies that coincide in all but n states. In these states each policy chooses one of two possible actions. We show that the stationary distributions of n+1 such policies uniquely determine the stationary distributions of all other such policies. An explicit formula for calculation is given.
Create a lesson
Related papers
Almost sure exponential stability of multi-dimensional stochastic systems driven by a symmetric α-stable process
Xuekang Zhang, Hongjian Liu, Guangjun Shen
The Arc Threshold of the Two-Row Ising Torus: Exact Factorization, an Algebraic Critical Coupling, Uniformity in the Circumference, and the Rate of Convergence
Marco Mandap
Markovian Lifts and Multifactor Approximations for Weighted Sub-fractional Brownian Motion
J. H. Ramirez-Gonzalez, Edison Cuba
Rényi mixing bounds for kinetic samplers via L2 hypocoercivity
Jianfeng Lu, Yinchen Luo
Bayesian Filters via the Stochastic Category
Kenichi Bannai, Masahiro Kurisaki
Synchronization of Enriched Spin Glasses
Franziska Engler, Justin Ko