Stationary and Non-Stationary Transition Probabilities in Decision Making: Modeling COVID-19 Dynamics

Abstract

This study introduces a comparative modeling framework using stationary and non-stationary transition probabilities within a Markov Decision Process (MDP) to assess COVID-19 disease dynamics. Stationary transition probabilities assume constant transition rates, while non-stationary transitions reflect time-dependent behaviors including policy interventions or behavioral changes. We develop a comprehensive compartmental model with transitions based on binomial and multinomial processes. Mathematical models for both stationary and non-stationary transition frameworks are developed and simulated over a 365-day period to emphasize dynamic variations in epidemic outcomes. Our findings highlight the significance of non-stationary modeling in accurately representing the dynamic characteristics of pandemic situations and provide recommendations for optimizing public health interventions under uncertainty. This comparative analysis offers useful information for epidemiological modeling and decision-making in dynamic risk environments.