Mean-Variance Type Controls Involving a Hidden Markov Chain: Models and Numerical Approximation

Abstract

Motivated by applications arising in networked systems, this work examines controlled regime-switching systems that stem from a mean-variance formulation. A main point is that the switching process is a hidden Markov chain. An additional piece of information, namely, a noisy observation of switching process corrupted by white noise is available. We focus on minimizing the variance subject to a fixed terminal expectation. Using the Wonham filter, we convert the partially observed system to a completely observable one first. Since closed-form solutions are virtually impossible be obtained, a Markov chain approximation method is used to devise a computational scheme. Convergence of the algorithm is obtained. A numerical example is provided to demonstrate the results.