A New Method for Computing Stationary Distribution and Steady-State Performance Measures of a Continuous-State Markov Chain with a Queuing Application

Abstract

Applications of stochastic models often involve the evaluation of steady-state performance, which requires solving a set of balance equations. In most cases of interest, the number of equations is infinite or even uncountable. As a result, numerical or analytical solutions are unavailable. This is true even when the system state is one-dimensional. This paper develops a general method for computing stationary distributions and steady-state performance measures of stochastic systems that can be described as continuous-state Markov chains supported on R. The balance equations are numerically solved by properly constructing a proxy Markov chain with finite states. We show the consistency of the approximate solution and provide deterministic non-asymptotic error bounds under the supremum norm. Our finite approximation method is near-optimal among all approximation methods using discrete distributions, including the empirical distributions generated by a simulation approach. We apply the developed method to compute the stationary distribution of virtual waiting time in a G/G/1+G queue and associated performance measures under certain mild but general differentiability and boundedness assumptions on the inter-arrival, service, and patience time distributions. Numerical experiments validate the accuracy and efficiency of our method, and show it outperforms a standard Markov chain Monte Carlo method by several orders of magnitude. The developed method is also significantly more accurate than the available fluid approximations for this queue.