Age-based maintenance under population heterogeneity: Optimal exploration and exploitation

We consider a system with a finite lifespan and a single critical component that is subject to random failures. An age-based replacement policy is applied to preventively replace the component before its failure. The components used for replacement come from either a weak population or a strong population, referred to as population heterogeneity. However, the true population type is unknown to the decision maker. By considering that the decision maker has a belief on the probability of having a weak population, we build a partially observable Markov decision process model with the objective of minimizing the total cost over the lifespan of the system. The resulting optimal policy updates the belief variable in a Bayesian fashion by using the data obtained over the course of the system lifespan, and it denotes when to execute preventive replacement. It optimally balances the trade-off between the cost of learning the true population type (via deliberately delaying the preventive replacement time to better learn the population type) and the cost of maintenance activities. By addressing this so-called exploration-exploitation trade-off, we generate insights on the optimal policy and compare its performance with existing heuristic approaches from the literature. We also characterize a lower bound to the optimal cost, allowing us to determine the value of resolving the uncertainty on the population type.