IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v203y2020ics0951832020305950.html
   My bibliography  Save this article

Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks

Author

Listed:
  • Zhang, Nailong
  • Si, Wujun

Abstract

Condition-Based Maintenance (CBM) planning for multi-component systems has been receiving increasing attention in recent years. Most existing research on CBM assumes that preventive maintenances should be conducted when the degradations of system components reach specific threshold levels upon inspection. However, the search of optimal maintenance threshold levels is often efficient for low-dimensional CBM but becomes challenging if the number of components gets large, especially when those components are subject to complex dependencies. To overcome the challenge, in this paper we propose a novel and flexible CBM model based on a customized deep reinforcement learning for multi-component systems with dependent competing risks. Both stochastic and economic dependencies among the components are considered. Specifically, different from the threshold-based decision making paradigm used in traditional CBM, the proposed model directly maps the multi-component degradation measurements at each inspection epoch to the maintenance decision space with a cost minimization objective, and the leverage of deep reinforcement learning enables high computational efficiencies and thus makes the proposed model suitable for both low and high dimensional CBM. Various numerical studies are conducted for model validations.

Suggested Citation

  • Zhang, Nailong & Si, Wujun, 2020. "Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks," Reliability Engineering and System Safety, Elsevier, vol. 203(C).
  • Handle: RePEc:eee:reensy:v:203:y:2020:i:c:s0951832020305950
    DOI: 10.1016/j.ress.2020.107094
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832020305950
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ress.2020.107094?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tian, Zhigang & Jin, Tongdan & Wu, Bairong & Ding, Fangfang, 2011. "Condition based maintenance optimization for wind power generation systems under continuous monitoring," Renewable Energy, Elsevier, vol. 36(5), pages 1502-1509.
    2. de Jonge, Bram & Teunter, Ruud & Tinga, Tiedo, 2017. "The influence of practical factors on the benefits of condition-based maintenance over time-based maintenance," Reliability Engineering and System Safety, Elsevier, vol. 158(C), pages 21-30.
    3. Li, Heping & Deloux, Estelle & Dieulle, Laurence, 2016. "A condition-based maintenance policy for multi-component systems with Lévy copulas dependence," Reliability Engineering and System Safety, Elsevier, vol. 149(C), pages 44-55.
    4. Alaswad, Suzan & Xiang, Yisha, 2017. "A review on condition-based maintenance optimization models for stochastically deteriorating system," Reliability Engineering and System Safety, Elsevier, vol. 157(C), pages 54-63.
    5. Chen, Nan & Ye, Zhi-Sheng & Xiang, Yisha & Zhang, Linmiao, 2015. "Condition-based maintenance using the inverse Gaussian degradation model," European Journal of Operational Research, Elsevier, vol. 243(1), pages 190-199.
    6. Wu, Jingda & He, Hongwen & Peng, Jiankun & Li, Yuecheng & Li, Zhanjiang, 2018. "Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus," Applied Energy, Elsevier, vol. 222(C), pages 799-811.
    7. Verbert, K. & De Schutter, B. & Babuška, R., 2017. "Timely condition-based maintenance planning for multi-component systems," Reliability Engineering and System Safety, Elsevier, vol. 159(C), pages 310-321.
    8. Ding, Fangfang & Tian, Zhigang, 2012. "Opportunistic maintenance for wind farms considering multi-level imperfect maintenance thresholds," Renewable Energy, Elsevier, vol. 45(C), pages 175-182.
    9. Nailong Zhang & Qingyu Yang, 2015. "Optimal maintenance planning for repairable multi-component systems subject to dependent competing risks," IISE Transactions, Taylor & Francis Journals, vol. 47(5), pages 521-532, May.
    10. Liu, Bin & Wu, Shaomin & Xie, Min & Kuo, Way, 2017. "A condition-based maintenance policy for degrading systems with age- and state-dependent operating cost," European Journal of Operational Research, Elsevier, vol. 263(3), pages 879-887.
    11. Van Horenbeek, Adriaan & Pintelon, Liliane, 2013. "A dynamic predictive maintenance policy for complex multi-component systems," Reliability Engineering and System Safety, Elsevier, vol. 120(C), pages 39-50.
    12. Do, Phuc & Assaf, Roy & Scarf, Phil & Iung, Benoit, 2019. "Modelling and application of condition-based maintenance for a two-component system with stochastic and economic dependencies," Reliability Engineering and System Safety, Elsevier, vol. 182(C), pages 86-97.
    13. Olde Keizer, Minou C.A. & Flapper, Simme Douwe P. & Teunter, Ruud H., 2017. "Condition-based maintenance policies for systems with multiple dependent components: A review," European Journal of Operational Research, Elsevier, vol. 261(2), pages 405-420.
    14. Cheng, Tianjin & Pandey, Mahesh D. & van der Weide, J.A.M., 2012. "The probability distribution of maintenance cost of a system affected by the gamma process of degradation: Finite time solution," Reliability Engineering and System Safety, Elsevier, vol. 108(C), pages 65-76.
    15. Shahraki, Ameneh Forouzandeh & Yadav, Om Prakash & Vogiatzis, Chrysafis, 2020. "Selective maintenance optimization for multi-state systems considering stochastically dependent components and stochastic imperfect maintenance actions," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    16. Tian, Zhigang & Liao, Haitao, 2011. "Condition based maintenance optimization for multi-component systems using proportional hazards model," Reliability Engineering and System Safety, Elsevier, vol. 96(5), pages 581-589.
    17. Rasmekomen, Nipat & Parlikad, Ajith Kumar, 2016. "Condition-based maintenance of multi-component systems with degradation state-rate interactions," Reliability Engineering and System Safety, Elsevier, vol. 148(C), pages 1-10.
    18. Dieulle, L. & Berenguer, C. & Grall, A. & Roussignol, M., 2003. "Sequential condition-based maintenance scheduling for a deteriorating system," European Journal of Operational Research, Elsevier, vol. 150(2), pages 451-461, October.
    19. Liao, Haitao & Elsayed, Elsayed A. & Chan, Ling-Yau, 2006. "Maintenance of continuously monitored degrading systems," European Journal of Operational Research, Elsevier, vol. 175(2), pages 821-835, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. de Jonge, Bram & Scarf, Philip A., 2020. "A review on maintenance optimization," European Journal of Operational Research, Elsevier, vol. 285(3), pages 805-824.
    2. Liu, Xingchen & Sun, Qiuzhuang & Ye, Zhi-Sheng & Yildirim, Murat, 2021. "Optimal multi-type inspection policy for systems with imperfect online monitoring," Reliability Engineering and System Safety, Elsevier, vol. 207(C).
    3. Zhang, Nan & Cai, Kaiquan & Zhang, Jun & Wang, Tian, 2022. "A condition-based maintenance policy considering failure dependence and imperfect inspection for a two-component system," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    4. Alaswad, Suzan & Xiang, Yisha, 2017. "A review on condition-based maintenance optimization models for stochastically deteriorating system," Reliability Engineering and System Safety, Elsevier, vol. 157(C), pages 54-63.
    5. Li, Heping & Zhu, Wenjin & Dieulle, Laurence & Deloux, Estelle, 2022. "Condition-based maintenance strategies for stochastically dependent systems using Nested Lévy copulas," Reliability Engineering and System Safety, Elsevier, vol. 217(C).
    6. Wang, Yukun & Li, Xiaopeng & Chen, Junyan & Liu, Yiliu, 2022. "A condition-based maintenance policy for multi-component systems subject to stochastic and economic dependencies," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    7. Liu, Bin & Pandey, Mahesh D. & Wang, Xiaolin & Zhao, Xiujie, 2021. "A finite-horizon condition-based maintenance policy for a two-unit system with dependent degradation processes," European Journal of Operational Research, Elsevier, vol. 295(2), pages 705-717.
    8. Shi, Yue & Zhu, Weihang & Xiang, Yisha & Feng, Qianmei, 2020. "Condition-based maintenance optimization for multi-component systems subject to a system reliability requirement," Reliability Engineering and System Safety, Elsevier, vol. 202(C).
    9. Uit Het Broek, Michiel A.J. & Teunter, Ruud H. & de Jonge, Bram & Veldman, Jasper, 2021. "Joint condition-based maintenance and load-sharing optimization for two-unit systems with economic dependency," European Journal of Operational Research, Elsevier, vol. 295(3), pages 1119-1131.
    10. Wang, Jun & Zhu, Xiaoyan, 2021. "Joint optimization of condition-based maintenance and inventory control for a k-out-of-n:F system of multi-state degrading components," European Journal of Operational Research, Elsevier, vol. 290(2), pages 514-529.
    11. Nguyen, Kim-Anh & Do, Phuc & Grall, Antoine, 2017. "Joint predictive maintenance and inventory strategy for multi-component systems using Birnbaum’s structural importance," Reliability Engineering and System Safety, Elsevier, vol. 168(C), pages 249-261.
    12. Shahraki, Ameneh Forouzandeh & Yadav, Om Prakash & Vogiatzis, Chrysafis, 2020. "Selective maintenance optimization for multi-state systems considering stochastically dependent components and stochastic imperfect maintenance actions," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    13. Wu, Tianyi & Yang, Li & Ma, Xiaobing & Zhang, Zihan & Zhao, Yu, 2020. "Dynamic maintenance strategy with iteratively updated group information," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    14. Olde Keizer, Minou C.A. & Flapper, Simme Douwe P. & Teunter, Ruud H., 2017. "Condition-based maintenance policies for systems with multiple dependent components: A review," European Journal of Operational Research, Elsevier, vol. 261(2), pages 405-420.
    15. Dias, Luis & Leitão, Armando & Guimarães, Luis, 2021. "Resource definition and allocation for a multi-asset portfolio with heterogeneous degradation," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    16. Aizpurua, J.I. & Catterson, V.M. & Papadopoulos, Y. & Chiacchio, F. & D'Urso, D., 2017. "Supporting group maintenance through prognostics-enhanced dynamic dependability prediction," Reliability Engineering and System Safety, Elsevier, vol. 168(C), pages 171-188.
    17. Giovanni Rinaldi & Philipp R. Thies & Lars Johanning, 2021. "Current Status and Future Trends in the Operation and Maintenance of Offshore Wind Turbines: A Review," Energies, MDPI, vol. 14(9), pages 1-28, April.
    18. KarabaÄŸ, Oktay & Eruguz, Ayse Sena & Basten, Rob, 2020. "Integrated optimization of maintenance interventions and spare part selection for a partially observable multi-component system," Reliability Engineering and System Safety, Elsevier, vol. 200(C).
    19. Liu, Xinbao & Yang, Tianji & Pei, Jun & Liao, Haitao & Pohl, Edward A., 2019. "Replacement and inventory control for a multi-customer product service system with decreasing replacement costs," European Journal of Operational Research, Elsevier, vol. 273(2), pages 561-574.
    20. Yang, Li & Ye, Zhi-sheng & Lee, Chi-Guhn & Yang, Su-fen & Peng, Rui, 2019. "A two-phase preventive maintenance policy considering imperfect repair and postponed replacement," European Journal of Operational Research, Elsevier, vol. 274(3), pages 966-977.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:203:y:2020:i:c:s0951832020305950. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.