Maintenance Optimization for Asset Networks with Unknown Degradation Parameters

Abstract

We consider the key practical challenge of multi-asset maintenance optimization in settings where degradation parameters are heterogeneous and unknown, and must be inferred from degradation data. To address this, we propose scalable methods suitable for complex asset networks. Degradation is modeled as a stochastic shock process, and real-time data are continuously incorporated into estimation of shock rates and magnitudes via a Bayesian framework. This constitutes a partially observable Markov decision process formulation, from which we analytically derive monotonic policy structures. Moreover, we propose an open-loop feedback approach that enables policies trained via deep reinforcement learning (DRL) in a simulation environment with access to the true parameters to remain effective when deployed with real-time Bayesian point estimates instead. Complementing this, we develop a Bayesian Markov decision process (BMDP) framework wherein the agent maintains and updates posterior distributions during deployment. This formulation captures the evolution of parameter uncertainty over time, thereby facilitating the training of scalable DRL-based policies that adapt as additional data become available. We validate our approach through experiments on synthetic asset networks and a real-world case involving interventional X-ray system filaments. We find that the proposed DRL methods consistently outperform traditional heuristics across various scenarios. The policies trained for the BMDP perform well even when priors must be estimated from historical data, and remain effective in networks with high asset heterogeneity. Knowledge of true degradation parameters yields only marginal cost benefits, underscoring the ability of our approach to make effective decisions under limited information on degradation processes.