IDEAS home Printed from https://ideas.repec.org/a/wly/navres/v67y2020i7p548-558.html
   My bibliography  Save this article

Application of Markov renewal theory and semi‐Markov decision processes in maintenance modeling and optimization of multi‐unit systems

Author

Listed:
  • Nooshin Salari
  • Viliam Makis

Abstract

In this paper, a condition‐based maintenance model for a multi‐unit production system is proposed and analyzed using Markov renewal theory. The units of the system are subject to gradual deterioration, and the gradual deterioration process of each unit is described by a three‐state continuous time homogeneous Markov chain with two working states and a failure state. The production rate of the system is influenced by the deterioration process and the demand is constant. The states of the units are observable through regular inspections and the decision to perform maintenance depends on the number of units in each state. The objective is to obtain the steady‐state characteristics and the formula for the long‐run average cost for the controlled system. The optimal policy is obtained using a dynamic programming algorithm. The result is validated using a semi‐Markov decision process formulation and the policy iteration algorithm. Moreover, an analytical expression is obtained for the calculation of the mean time to initiate maintenance using the first passage time theory.

Suggested Citation

  • Nooshin Salari & Viliam Makis, 2020. "Application of Markov renewal theory and semi‐Markov decision processes in maintenance modeling and optimization of multi‐unit systems," Naval Research Logistics (NRL), John Wiley & Sons, vol. 67(7), pages 548-558, October.
    • Handle: RePEc:wly:navres:v:67:y:2020:i:7:p:548-558
    DOI: 10.1002/nav.21932
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/nav.21932
    Download Restriction: no
    File URL: https://libkey.io/10.1002/nav.21932?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Tian, Zhigang & Jin, Tongdan & Wu, Bairong & Ding, Fangfang, 2011. "Condition based maintenance optimization for wind power generation systems under continuous monitoring," Renewable Energy, Elsevier, vol. 36(5), pages 1502-1509.
    2. Xiaohong Zhang & Jianchao Zeng, 2015. "Deterioration state space partitioning method for opportunistic maintenance modelling of identical multi-unit systems," International Journal of Production Research, Taylor & Francis Journals, vol. 53(7), pages 2100-2118, April.
    3. Michael Q. Anderson, 1981. "Monotone optimal preventive maintenance policies for stochastically failing equipment," Naval Research Logistics Quarterly, John Wiley & Sons, vol. 28(3), pages 347-358, September.
    4. Tang, Diyin & Makis, Viliam & Jafari, Leila & Yu, Jinsong, 2015. "Optimal maintenance policy and residual life estimation for a slowly degrading system subject to condition monitoring," Reliability Engineering and System Safety, Elsevier, vol. 134(C), pages 198-207.
    5. Poppe, Joeri & Boute, Robert N. & Lambrecht, Marc R., 2018. "A hybrid condition-based maintenance policy for continuously monitored components with two degradation thresholds," European Journal of Operational Research, Elsevier, vol. 268(2), pages 515-532.
    6. William S. Jewell, 1963. "Markov-Renewal Programming. II: Infinite Return Models, Example," Operations Research, INFORMS, vol. 11(6), pages 949-971, December.
    7. Xiaodong Yao & Xiaolan Xie & Michael C. Fu & Steven I. Marcus, 2005. "Optimal joint preventive maintenance and production policies," Naval Research Logistics (NRL), John Wiley & Sons, vol. 52(7), pages 668-681, October.
    8. Olde Keizer, Minou C.A. & Flapper, Simme Douwe P. & Teunter, Ruud H., 2017. "Condition-based maintenance policies for systems with multiple dependent components: A review," European Journal of Operational Research, Elsevier, vol. 261(2), pages 405-420.
    9. Morton Klein, 1962. "Inspection--Maintenance--Replacement Schedules Under Markovian Deterioration," Management Science, INFORMS, vol. 9(1), pages 25-32, October.
    10. John S. de Cani, 1964. "A Dynamic Programming Algorithm for Embedded Markov Chains when the Planning Horizon is at Infinity," Management Science, INFORMS, vol. 10(4), pages 716-733, July.
    11. William S. Jewell, 1963. "Markov-Renewal Programming. I: Formulation, Finite Return Models," Operations Research, INFORMS, vol. 11(6), pages 938-948, December.
    12. Xiao Liu & Jingrui Li & Khalifa Al-Khalifa & Abdelmagid Hamouda & David Coit & Elsayed Elsayed, 2013. "Condition-based maintenance for continuously monitored degrading systems with multiple failure modes," IISE Transactions, Taylor & Francis Journals, vol. 45(4), pages 422-435.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Najafi, Seyedvahid & Lee, Chi-Guhn, 2023. "A deep reinforcement learning approach for repair-based maintenance of multi-unit systems using proportional hazards model," Reliability Engineering and System Safety, Elsevier, vol. 234(C).
    2. Azizi, Fariba & Salari, Nooshin, 2023. "A novel condition-based maintenance framework for parallel manufacturing systems based on bivariate birth/birth–death processes," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    3. Cavalcante, Cristiano A.V. & Lopes, Rodrigo S. & Scarf, Philip A., 2021. "Inspection and replacement policy with a fixed periodic schedule," Reliability Engineering and System Safety, Elsevier, vol. 208(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alaswad, Suzan & Xiang, Yisha, 2017. "A review on condition-based maintenance optimization models for stochastically deteriorating system," Reliability Engineering and System Safety, Elsevier, vol. 157(C), pages 54-63.
    2. Wu, Tianyi & Yang, Li & Ma, Xiaobing & Zhang, Zihan & Zhao, Yu, 2020. "Dynamic maintenance strategy with iteratively updated group information," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    3. Liu, Xingchen & Sun, Qiuzhuang & Ye, Zhi-Sheng & Yildirim, Murat, 2021. "Optimal multi-type inspection policy for systems with imperfect online monitoring," Reliability Engineering and System Safety, Elsevier, vol. 207(C).
    4. Zhang, Nailong & Si, Wujun, 2020. "Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks," Reliability Engineering and System Safety, Elsevier, vol. 203(C).
    5. Liu, Gehui & Chen, Shaokuan & Jin, Hua & Liu, Shuang, 2021. "Optimum opportunistic maintenance schedule incorporating delay time theory with imperfect maintenance," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    6. Prasenjit Mondal, 2020. "Computing semi-stationary optimal policies for multichain semi-Markov decision processes," Annals of Operations Research, Springer, vol. 287(2), pages 843-865, April.
    7. Ece Zeliha Demirci & Joachim Arts & Geert-Jan Van Houtum, 2022. "A restless bandit approach for capacitated condition based maintenance scheduling," DEM Discussion Paper Series 22-01, Department of Economics at the University of Luxembourg.
    8. de Jonge, Bram & Scarf, Philip A., 2020. "A review on maintenance optimization," European Journal of Operational Research, Elsevier, vol. 285(3), pages 805-824.
    9. Wallace J. Hopp & Sung‐Chi Wu, 1988. "Multiaction maintenance under markovian deterioration and incomplete state information," Naval Research Logistics (NRL), John Wiley & Sons, vol. 35(5), pages 447-462, October.
    10. Zhang, Nan & Deng, Yingjun & Liu, Bin & Zhang, Jun, 2023. "Condition-based maintenance for a multi-component system in a dynamic operating environment," Reliability Engineering and System Safety, Elsevier, vol. 231(C).
    11. Chris P. Lee & Glenn M. Chertow & Stefanos A. Zenios, 2008. "Optimal Initiation and Management of Dialysis Therapy," Operations Research, INFORMS, vol. 56(6), pages 1428-1449, December.
    12. Duan, Chaoqun & Gong, Ting & Yan, Liangwen & Li, Xinmin, 2024. "Bi-level corrected residual life-based maintenance for deteriorating systems under competing risks," Reliability Engineering and System Safety, Elsevier, vol. 247(C).
    13. Olde Keizer, Minou C.A. & Flapper, Simme Douwe P. & Teunter, Ruud H., 2017. "Condition-based maintenance policies for systems with multiple dependent components: A review," European Journal of Operational Research, Elsevier, vol. 261(2), pages 405-420.
    14. Zhang, Xiaohong & Liao, Haitao & Zeng, Jianchao & Shi, Guannan & Zhao, Bing, 2021. "Optimal Condition-based Opportunistic Maintenance and Spare Parts Provisioning for a Two-unit System using a State Space Partitioning Approach," Reliability Engineering and System Safety, Elsevier, vol. 209(C).
    15. Hu, Jiawen & Chen, Piao, 2020. "Predictive maintenance of systems subject to hard failure based on proportional hazards model," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    16. Roy Assaf & Phuc Do & Samia Nefti-Meziani & Philip Scarf, 2018. "Wear rate–state interactions within a multi-component system: a study of a gearbox-accelerated life testing platform," Journal of Risk and Reliability, , vol. 232(4), pages 425-434, August.
    17. Azar, Kamyar & Hajiakhondi-Meybodi, Zohreh & Naderkhani, Farnoosh, 2022. "Semi-supervised clustering-based method for fault diagnosis and prognosis: A case study," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    18. Zhang, Yunzheng & Zhang, Xiaohong & Zeng, Jianchao & Wang, Jinhe & Xue, Songdong, 2019. "Lessees’ satisfaction and optimal condition-based maintenance policy for leased system," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    19. Asadzadeh, S.M. & Azadeh, A., 2014. "An integrated systemic model for optimization of condition-based maintenance with human error," Reliability Engineering and System Safety, Elsevier, vol. 124(C), pages 117-131.
    20. Lodewijk Kallenberg, 2013. "Derman’s book as inspiration: some results on LP for MDPs," Annals of Operations Research, Springer, vol. 208(1), pages 63-94, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:navres:v:67:y:2020:i:7:p:548-558. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)1520-6750 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.