Joint optimization of preventive maintenance and production scheduling for multi-state production systems based on reinforcement learning

Preventive maintenance and production scheduling are two important and interactive activities in production systems. In this work, the integrated optimization problem of production scheduling for multi-state single-machine production systems experiencing degradation processes is investigated. Preventive maintenance tasks and jobs scheduling are jointly considered to find the optimal production policy by considering the processing costs, the maintenance costs, and the completion rewards, simultaneously. We formulate the integrated optimization problem as Markov decision process framework. R-learning algorithm is introduced to maximize the long-run expected average rewards per time unit over infinite horizon. On the basis of the analysis of the optimal stationary policy, the appropriate condition to perform preventive maintenance following optimal stationary policy is presented. This provides the basis for the improvement in R-learning algorithm. Furthermore, a novel heuristic reinforcement learning method is proposed to deal with the integrated model more efficiently. Finally, we present the simulation results and analysis of the proposed algorithm's performance in terms of the number of job types and machine states. The simulation results and analysis show the effectiveness of the proposed approach for solving the integrated problems.