IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v233y2023ics095183202300008x.html
   My bibliography  Save this article

A deep reinforcement learning method for structural dominant failure modes searching based on self-play strategy

Author

Listed:
  • Guan, Xiaoshu
  • Sun, Huabin
  • Hou, Rongrong
  • Xu, Yang
  • Bao, Yuequan
  • Li, Hui

Abstract

In the research area of structural reliability analysis (SRA), the dominant failure modes (DFMs) of a structural system make significant contributions to life-span failure prediction and safety assessment. However, the high computational cost caused by the combinatorial explosion is the main problem in DFMs searching that hinders its application and further development. Recently, many successful applications have proved that the self-play deep reinforcement learning (DRL) has a strong ability to obtain action policy in the face of combinatorial explosion problems. Inspired by this, a self-play strategy is designed to optimize the DRL-based DFMs searching process and reduce the computational effort. A scoring function is designed and used as the refereeing standard of the self-play games and helps improve the efficiency of Monte Carlo tree search (MCTS) in an asynchronous training process. In comparison with the β-unzipping method and exploration-based DFMs searching method, the proposed method significantly improved training efficiency with an accuracy of over 95% and a lower requirement of the number of finite element analysis (FEA), both of which contribute to the policy learning of failure component selection. In summary, the method shows potential applications for actual structures and makes valuable contributions to the problem with high computing costs.

Suggested Citation

  • Guan, Xiaoshu & Sun, Huabin & Hou, Rongrong & Xu, Yang & Bao, Yuequan & Li, Hui, 2023. "A deep reinforcement learning method for structural dominant failure modes searching based on self-play strategy," Reliability Engineering and System Safety, Elsevier, vol. 233(C).
  • Handle: RePEc:eee:reensy:v:233:y:2023:i:c:s095183202300008x
    DOI: 10.1016/j.ress.2023.109093
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S095183202300008X
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ress.2023.109093?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhou, Changcong & Zhang, Hanlin & Valdebenito, Marcos A. & Zhao, Haodong, 2022. "A general hierarchical ensemble-learning framework for structural reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    2. Jiang, Chen & Qiu, Haobo & Gao, Liang & Wang, Dapeng & Yang, Zan & Chen, Liming, 2020. "EEK-SYS: System reliability analysis through estimation error-guided adaptive Kriging approximation of multiple limit state surfaces," Reliability Engineering and System Safety, Elsevier, vol. 198(C).
    3. Zhang, Xufang & Wang, Lei & Sørensen, John Dalsgaard, 2019. "REIF: A novel active-learning function toward adaptive Kriging surrogate models for structural reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 185(C), pages 440-454.
    4. Lim, HyeongUk & Manuel, Lance, 2021. "Distribution-free polynomial chaos expansion surrogate models for efficient structural reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 205(C).
    5. Fauriat, W. & Gayton, N., 2014. "AK-SYS: An adaptation of the AK-MCS method for system reliability," Reliability Engineering and System Safety, Elsevier, vol. 123(C), pages 137-144.
    6. Andriotis, C.P. & Papakonstantinou, K.G., 2019. "Managing engineering systems with large state and action spaces through deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    7. Xu, Zhaoyi & Saleh, Joseph Homer, 2021. "Machine learning for reliability engineering and safety applications: Review of current status and future opportunities," Reliability Engineering and System Safety, Elsevier, vol. 211(C).
    8. Schwalbe, Ulrich & Walker, Paul, 2001. "Zermelo and the Early History of Game Theory," Games and Economic Behavior, Elsevier, vol. 34(1), pages 123-137, January.
    9. Zhang, Kun & Chen, Ning & Zeng, Peng & Liu, Jian & Beer, Michael, 2022. "An efficient reliability analysis method for structures with hybrid time-dependent uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    10. Huang, Shi-Ya & Zhang, Shao-He & Liu, Lei-Lei, 2022. "A new active learning Kriging metamodel for structural system reliability analysis with multiple failure modes," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    11. David Silver & Julian Schrittwieser & Karen Simonyan & Ioannis Antonoglou & Aja Huang & Arthur Guez & Thomas Hubert & Lucas Baker & Matthew Lai & Adrian Bolton & Yutian Chen & Timothy Lillicrap & Fan , 2017. "Mastering the game of Go without human knowledge," Nature, Nature, vol. 550(7676), pages 354-359, October.
    12. Guan, Xiaoshu & Xiang, Zhengliang & Bao, Yuequan & Li, Hui, 2022. "Structural dominant failure modes searching method based on deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    13. Zhang, Nailong & Si, Wujun, 2020. "Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks," Reliability Engineering and System Safety, Elsevier, vol. 203(C).
    14. Zhou, Yicheng & Lu, Zhenzhou & Yun, Wanying, 2020. "Active sparse polynomial chaos expansion for system reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 202(C).
    15. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    16. Xiang, Zhengliang & Bao, Yuequan & Tang, Zhiyi & Li, Hui, 2020. "Deep reinforcement learning-based sampling method for structural reliability assessment," Reliability Engineering and System Safety, Elsevier, vol. 199(C).
    17. Bao, Yuequan & Xiang, Zhengliang & Li, Hui, 2021. "Adaptive subset searching-based deep neural network method for structural reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    18. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    19. Saraygord Afshari, Sajad & Enayatollahi, Fatemeh & Xu, Xiangyang & Liang, Xihui, 2022. "Machine learning-based methods in structural reliability analysis: A review," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tian, Yuxuan & Guan, Xiaoshu & Sun, Huabin & Bao, Yuequan, 2024. "An adaptive structural dominant failure modes searching method based on graph neural network," Reliability Engineering and System Safety, Elsevier, vol. 243(C).
    2. den Heijer, Frank & Kok, Matthijs, 2024. "Risk-based portfolio planning of dike reinforcements," Reliability Engineering and System Safety, Elsevier, vol. 242(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guan, Xiaoshu & Xiang, Zhengliang & Bao, Yuequan & Li, Hui, 2022. "Structural dominant failure modes searching method based on deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    2. Saraygord Afshari, Sajad & Enayatollahi, Fatemeh & Xu, Xiangyang & Liang, Xihui, 2022. "Machine learning-based methods in structural reliability analysis: A review," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    3. Zhang, Qin & Liu, Yu & Xiang, Yisha & Xiahou, Tangfan, 2024. "Reinforcement learning in reliability and maintenance optimization: A tutorial," Reliability Engineering and System Safety, Elsevier, vol. 251(C).
    4. Xu, Zhaoyi & Saleh, Joseph Homer, 2021. "Machine learning for reliability engineering and safety applications: Review of current status and future opportunities," Reliability Engineering and System Safety, Elsevier, vol. 211(C).
    5. Lee, Dongkyu & Song, Junho, 2023. "Risk-informed operation and maintenance of complex lifeline systems using parallelized multi-agent deep Q-network," Reliability Engineering and System Safety, Elsevier, vol. 239(C).
    6. Morato, P.G. & Andriotis, C.P. & Papakonstantinou, K.G. & Rigo, P., 2023. "Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 235(C).
    7. Wang, Jian & Sun, Zhili & Cao, Runan, 2021. "An efficient and robust Kriging-based method for system reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    8. Zhao, Qiangqiang & Duan, Jinyan & Jia, Kang & Hong, Jun, 2025. "PLIC-FSR-SYS: System reliability analysis based on parallel learning of influential components with filtered sample region," Reliability Engineering and System Safety, Elsevier, vol. 253(C).
    9. Mohammadi, Reza & He, Qing, 2022. "A deep reinforcement learning approach for rail renewal and maintenance planning," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    10. Yao, Wen & Zheng, Xiaohu & Zhang, Jun & Wang, Ning & Tang, Guijian, 2023. "Deep adaptive arbitrary polynomial chaos expansion: A mini-data-driven semi-supervised method for uncertainty quantification," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    11. Phan, Hieu Chi & Dhar, Ashutosh Sutra & Bui, Nang Duc, 2023. "Reliability assessment of pipelines crossing strike-slip faults considering modeling uncertainties using ANN models," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    12. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    13. Zhang, Yihao & Chai, Zhaojie & Lykotrafitis, George, 2021. "Deep reinforcement learning with a particle dynamics environment applied to emergency evacuation of a room with obstacles," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    14. Weifan Long & Taixian Hou & Xiaoyi Wei & Shichao Yan & Peng Zhai & Lihua Zhang, 2023. "A Survey on Population-Based Deep Reinforcement Learning," Mathematics, MDPI, vol. 11(10), pages 1-17, May.
    15. Yifeng Guo & Xingyu Fu & Yuyan Shi & Mingwen Liu, 2018. "Robust Log-Optimal Strategy with Reinforcement Learning," Papers 1805.00205, arXiv.org.
    16. Xie, Haipeng & Tang, Lingfeng & Zhu, Hao & Cheng, Xiaofeng & Bie, Zhaohong, 2023. "Robustness assessment and enhancement of deep reinforcement learning-enabled load restoration for distribution systems," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    17. Wang, Yanzhong & Xie, Bin & E, Shiyuan, 2022. "Adaptive relevance vector machine combined with Markov-chain-based importance sampling for reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 220(C).
    18. Huang, Shi-Ya & Zhang, Shao-He & Liu, Lei-Lei, 2022. "A new active learning Kriging metamodel for structural system reliability analysis with multiple failure modes," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    19. Yuchao Dong, 2022. "Randomized Optimal Stopping Problem in Continuous time and Reinforcement Learning Algorithm," Papers 2208.02409, arXiv.org, revised Sep 2023.
    20. Shijun Wang & Baocheng Zhu & Chen Li & Mingzhe Wu & James Zhang & Wei Chu & Yuan Qi, 2020. "Riemannian Proximal Policy Optimization," Computer and Information Science, Canadian Center of Science and Education, vol. 13(3), pages 1-93, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:233:y:2023:i:c:s095183202300008x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.