Multistate Bayesian Control Chart Over a Finite Horizon

We study a multistate partially observable process control model with a general state transition structure. The process is initially in control and subject to Markovian deterioration that can bring it to out-of-control states. The process may continue making transitions among the out-of-control states, or even back to the in-control state until it reaches an absorbing state. We assume that at least one out-of-control state is absorbing. The objective is to minimize the expected total cost over a finite horizon. By transforming the standard Cartesian belief space into the spherical coordinate system, we show that the optimal policy has a simple control-limit structure. We also examine two specialized models. The first is the phase-type transition time model, in which we develop an algorithm whose complexity is not affected by the number of phases. The second is a model with multiple absorbing out-of-control states, by which we show that certain out-of-control states may incur less total cost than the in-control state, a phenomenon never occurs in the two-state models. We conclude that there are fundamental differences between multistate models and two-state models, and that the spherical coordinate transformation offers significant analytical and computational benefits.