IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v233y2023ics095183202300008x.html
   My bibliography  Save this article

A deep reinforcement learning method for structural dominant failure modes searching based on self-play strategy

Author

Listed:
  • Guan, Xiaoshu
  • Sun, Huabin
  • Hou, Rongrong
  • Xu, Yang
  • Bao, Yuequan
  • Li, Hui

Abstract

In the research area of structural reliability analysis (SRA), the dominant failure modes (DFMs) of a structural system make significant contributions to life-span failure prediction and safety assessment. However, the high computational cost caused by the combinatorial explosion is the main problem in DFMs searching that hinders its application and further development. Recently, many successful applications have proved that the self-play deep reinforcement learning (DRL) has a strong ability to obtain action policy in the face of combinatorial explosion problems. Inspired by this, a self-play strategy is designed to optimize the DRL-based DFMs searching process and reduce the computational effort. A scoring function is designed and used as the refereeing standard of the self-play games and helps improve the efficiency of Monte Carlo tree search (MCTS) in an asynchronous training process. In comparison with the β-unzipping method and exploration-based DFMs searching method, the proposed method significantly improved training efficiency with an accuracy of over 95% and a lower requirement of the number of finite element analysis (FEA), both of which contribute to the policy learning of failure component selection. In summary, the method shows potential applications for actual structures and makes valuable contributions to the problem with high computing costs.

Suggested Citation

  • Guan, Xiaoshu & Sun, Huabin & Hou, Rongrong & Xu, Yang & Bao, Yuequan & Li, Hui, 2023. "A deep reinforcement learning method for structural dominant failure modes searching based on self-play strategy," Reliability Engineering and System Safety, Elsevier, vol. 233(C).
  • Handle: RePEc:eee:reensy:v:233:y:2023:i:c:s095183202300008x
    DOI: 10.1016/j.ress.2023.109093
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S095183202300008X
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ress.2023.109093?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhou, Changcong & Zhang, Hanlin & Valdebenito, Marcos A. & Zhao, Haodong, 2022. "A general hierarchical ensemble-learning framework for structural reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    2. Jiang, Chen & Qiu, Haobo & Gao, Liang & Wang, Dapeng & Yang, Zan & Chen, Liming, 2020. "EEK-SYS: System reliability analysis through estimation error-guided adaptive Kriging approximation of multiple limit state surfaces," Reliability Engineering and System Safety, Elsevier, vol. 198(C).
    3. Zhang, Xufang & Wang, Lei & Sørensen, John Dalsgaard, 2019. "REIF: A novel active-learning function toward adaptive Kriging surrogate models for structural reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 185(C), pages 440-454.
    4. Zhang, Nailong & Si, Wujun, 2020. "Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks," Reliability Engineering and System Safety, Elsevier, vol. 203(C).
    5. Zhou, Yicheng & Lu, Zhenzhou & Yun, Wanying, 2020. "Active sparse polynomial chaos expansion for system reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 202(C).
    6. Lim, HyeongUk & Manuel, Lance, 2021. "Distribution-free polynomial chaos expansion surrogate models for efficient structural reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 205(C).
    7. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    8. Fauriat, W. & Gayton, N., 2014. "AK-SYS: An adaptation of the AK-MCS method for system reliability," Reliability Engineering and System Safety, Elsevier, vol. 123(C), pages 137-144.
    9. Andriotis, C.P. & Papakonstantinou, K.G., 2019. "Managing engineering systems with large state and action spaces through deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    10. Xu, Zhaoyi & Saleh, Joseph Homer, 2021. "Machine learning for reliability engineering and safety applications: Review of current status and future opportunities," Reliability Engineering and System Safety, Elsevier, vol. 211(C).
    11. Xiang, Zhengliang & Bao, Yuequan & Tang, Zhiyi & Li, Hui, 2020. "Deep reinforcement learning-based sampling method for structural reliability assessment," Reliability Engineering and System Safety, Elsevier, vol. 199(C).
    12. Bao, Yuequan & Xiang, Zhengliang & Li, Hui, 2021. "Adaptive subset searching-based deep neural network method for structural reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 213(C).
    13. Schwalbe, Ulrich & Walker, Paul, 2001. "Zermelo and the Early History of Game Theory," Games and Economic Behavior, Elsevier, vol. 34(1), pages 123-137, January.
    14. Zhang, Kun & Chen, Ning & Zeng, Peng & Liu, Jian & Beer, Michael, 2022. "An efficient reliability analysis method for structures with hybrid time-dependent uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    15. Huang, Shi-Ya & Zhang, Shao-He & Liu, Lei-Lei, 2022. "A new active learning Kriging metamodel for structural system reliability analysis with multiple failure modes," Reliability Engineering and System Safety, Elsevier, vol. 228(C).
    16. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    17. David Silver & Julian Schrittwieser & Karen Simonyan & Ioannis Antonoglou & Aja Huang & Arthur Guez & Thomas Hubert & Lucas Baker & Matthew Lai & Adrian Bolton & Yutian Chen & Timothy Lillicrap & Fan , 2017. "Mastering the game of Go without human knowledge," Nature, Nature, vol. 550(7676), pages 354-359, October.
    18. Guan, Xiaoshu & Xiang, Zhengliang & Bao, Yuequan & Li, Hui, 2022. "Structural dominant failure modes searching method based on deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    19. Saraygord Afshari, Sajad & Enayatollahi, Fatemeh & Xu, Xiangyang & Liang, Xihui, 2022. "Machine learning-based methods in structural reliability analysis: A review," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tian, Yuxuan & Guan, Xiaoshu & Sun, Huabin & Bao, Yuequan, 2024. "An adaptive structural dominant failure modes searching method based on graph neural network," Reliability Engineering and System Safety, Elsevier, vol. 243(C).
    2. den Heijer, Frank & Kok, Matthijs, 2024. "Risk-based portfolio planning of dike reinforcements," Reliability Engineering and System Safety, Elsevier, vol. 242(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Saraygord Afshari, Sajad & Enayatollahi, Fatemeh & Xu, Xiangyang & Liang, Xihui, 2022. "Machine learning-based methods in structural reliability analysis: A review," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    2. Guan, Xiaoshu & Xiang, Zhengliang & Bao, Yuequan & Li, Hui, 2022. "Structural dominant failure modes searching method based on deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    3. Morato, P.G. & Andriotis, C.P. & Papakonstantinou, K.G. & Rigo, P., 2023. "Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning," Reliability Engineering and System Safety, Elsevier, vol. 235(C).
    4. Yao, Wen & Zheng, Xiaohu & Zhang, Jun & Wang, Ning & Tang, Guijian, 2023. "Deep adaptive arbitrary polynomial chaos expansion: A mini-data-driven semi-supervised method for uncertainty quantification," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    5. Xu, Zhaoyi & Saleh, Joseph Homer, 2021. "Machine learning for reliability engineering and safety applications: Review of current status and future opportunities," Reliability Engineering and System Safety, Elsevier, vol. 211(C).
    6. Lee, Dongkyu & Song, Junho, 2023. "Risk-informed operation and maintenance of complex lifeline systems using parallelized multi-agent deep Q-network," Reliability Engineering and System Safety, Elsevier, vol. 239(C).
    7. Wang, Jian & Sun, Zhili & Cao, Runan, 2021. "An efficient and robust Kriging-based method for system reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    8. Mohammadi, Reza & He, Qing, 2022. "A deep reinforcement learning approach for rail renewal and maintenance planning," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    9. Zhang, Yihao & Chai, Zhaojie & Lykotrafitis, George, 2021. "Deep reinforcement learning with a particle dynamics environment applied to emergency evacuation of a room with obstacles," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    10. Yifeng Guo & Xingyu Fu & Yuyan Shi & Mingwen Liu, 2018. "Robust Log-Optimal Strategy with Reinforcement Learning," Papers 1805.00205, arXiv.org.
    11. Xie, Haipeng & Tang, Lingfeng & Zhu, Hao & Cheng, Xiaofeng & Bie, Zhaohong, 2023. "Robustness assessment and enhancement of deep reinforcement learning-enabled load restoration for distribution systems," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    12. Zhao, Enyong & Wang, Qihan & Alamdari, Mehrisadat Makki & Gao, Wei, 2023. "Advanced virtual model assisted most probable point capturing method for engineering structures," Reliability Engineering and System Safety, Elsevier, vol. 239(C).
    13. Iwao Maeda & David deGraw & Michiharu Kitano & Hiroyasu Matsushima & Hiroki Sakaji & Kiyoshi Izumi & Atsuo Kato, 2020. "Deep Reinforcement Learning in Agent Based Financial Market Simulation," JRFM, MDPI, vol. 13(4), pages 1-17, April.
    14. Li, Wenqing & Ni, Shaoquan, 2022. "Train timetabling with the general learning environment and multi-agent deep reinforcement learning," Transportation Research Part B: Methodological, Elsevier, vol. 157(C), pages 230-251.
    15. Roy, Atin & Chakraborty, Subrata, 2023. "Support vector machine in structural reliability analysis: A review," Reliability Engineering and System Safety, Elsevier, vol. 233(C).
    16. Bo Hu & Jiaxi Li & Shuang Li & Jie Yang, 2019. "A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR," Energies, MDPI, vol. 12(19), pages 1-15, September.
    17. Yuan, Kai & Sui, Xi & Zhang, Shijie & Xiao, Ning-cong & Hu, Jinghan, 2024. "AK-SYS-IE: A novel adaptive Kriging-based method for system reliability assessment combining information entropy," Reliability Engineering and System Safety, Elsevier, vol. 246(C).
    18. Yang, Seonghyeok & Lee, Mingyu & Lee, Ikjin, 2023. "A new sampling approach for system reliability-based design optimization under multiple simulation models," Reliability Engineering and System Safety, Elsevier, vol. 231(C).
    19. Wang, Run-Zi & Gu, Hang-Hang & Zhu, Shun-Peng & Li, Kai-Shang & Wang, Ji & Wang, Xiao-Wei & Hideo, Miura & Zhang, Xian-Cheng & Tu, Shan-Tung, 2022. "A data-driven roadmap for creep-fatigue reliability assessment and its implementation in low-pressure turbine disk at elevated temperatures," Reliability Engineering and System Safety, Elsevier, vol. 225(C).
    20. De Moor, Bram J. & Gijsbrechts, Joren & Boute, Robert N., 2022. "Reward shaping to improve the performance of deep reinforcement learning in perishable inventory management," European Journal of Operational Research, Elsevier, vol. 301(2), pages 535-545.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:233:y:2023:i:c:s095183202300008x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.