Condition-based maintenance plan for multi-state systems using reinforcement learning

In this paper, a dynamic condition-based maintenance model is presented for a degrading system using a reinforcement learning approach. The system utilises a critical machine with multiple production states, which are estimated and obtained by some measurable indicators. Within the framework of the suggested maintenance policy, both the production state and machine virtual age are considered during the decision-making process for maintenance actions. The system is inspected at periodic times, and at each inspection, an action is selected from those that are available regarding spare parts and maintenance. Replacement and different levels of imperfect repairs are both considered. Additionally, the ordering of spare parts is also involved. The problem is formulated by using a discrete time Markov decision process and solved by the Q-learning algorithm. Numerical examples are provided to illustrate the proposed method. Also, the results indicate that the proposed method is time-saving and more cost-efficient than traditional methods.