IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v229y2023ics0951832022004306.html
   My bibliography  Save this article

Reliable neural networks for regression uncertainty estimation

Author

Listed:
  • Tohme, Tony
  • Vanslette, Kevin
  • Youcef-Toumi, Kamal

Abstract

While deep neural networks are highly performant and successful in a wide range of real-world problems, estimating their predictive uncertainty remains a challenging task. To address this challenge, we propose and implement a loss function for regression uncertainty estimation based on the Bayesian Validation Metric (BVM) framework while using ensemble learning. The proposed loss reproduces maximum likelihood estimation in the limiting case. A series of experiments on in-distribution data show that the proposed method is competitive with existing state-of-the-art methods. Experiments on out-of-distribution data show that the proposed method is robust to statistical change and exhibits superior predictive capability.

Suggested Citation

  • Tohme, Tony & Vanslette, Kevin & Youcef-Toumi, Kamal, 2023. "Reliable neural networks for regression uncertainty estimation," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
  • Handle: RePEc:eee:reensy:v:229:y:2023:i:c:s0951832022004306
    DOI: 10.1016/j.ress.2022.108811
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832022004306
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ress.2022.108811?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    2. Seo, Seung-Kwon & Yoon, Young-Gak & Lee, Ju-sung & Na, Jonggeol & Lee, Chul-Jin, 2022. "Deep Neural Network-based Optimization Framework for Safety Evacuation Route during Toxic Gas Leak Incidents," Reliability Engineering and System Safety, Elsevier, vol. 218(PA).
    3. Zhang, Weibin & Feng, Xinyu & Goerlandt, Floris & Liu, Qing, 2020. "Towards a Convolutional Neural Network model for classifying regional ship collision risk levels for waterway risk analysis," Reliability Engineering and System Safety, Elsevier, vol. 204(C).
    4. Zhou, Taotao & Han, Te & Droguett, Enrique Lopez, 2022. "Towards trustworthy machine fault diagnosis: A probabilistic Bayesian deep learning framework," Reliability Engineering and System Safety, Elsevier, vol. 224(C).
    5. Moradi, Ramin & Cofre-Martel, Sergio & Lopez Droguett, Enrique & Modarres, Mohammad & Groth, Katrina M., 2022. "Integration of deep learning and Bayesian networks for condition and operation risk monitoring of complex engineering systems," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    6. Nguyen, Khanh T.P. & Medjaher, Kamal & Gogu, Christian, 2022. "Probabilistic deep learning methodology for uncertainty quantification of remaining useful lifetime of multi-component systems," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    7. Izquierdo, J. & Crespo Márquez, A. & Uribetxebarria, J., 2019. "Dynamic artificial neural network-based reliability considering operational context of assets," Reliability Engineering and System Safety, Elsevier, vol. 188(C), pages 483-493.
    8. Zou, Qiling & Chen, Suren, 2021. "Resilience-based Recovery Scheduling of Transportation Network in Mixed Traffic Environment: A Deep-Ensemble-Assisted Active Learning Approach," Reliability Engineering and System Safety, Elsevier, vol. 215(C).
    9. Tohme, Tony & Vanslette, Kevin & Youcef-Toumi, Kamal, 2020. "A generalized Bayesian approach to model calibration," Reliability Engineering and System Safety, Elsevier, vol. 204(C).
    10. Zoubin Ghahramani, 2015. "Probabilistic machine learning and artificial intelligence," Nature, Nature, vol. 521(7553), pages 452-459, May.
    11. Liu, Di & Wang, Shaoping, 2021. "An artificial neural network supported stochastic process for degradation modeling and prediction," Reliability Engineering and System Safety, Elsevier, vol. 214(C).
    12. Saraygord Afshari, Sajad & Enayatollahi, Fatemeh & Xu, Xiangyang & Liang, Xihui, 2022. "Machine learning-based methods in structural reliability analysis: A review," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ebrahimi, Mehrdad & Nobahar, Elnaz & Mohammadi, Reza Karami & Noroozinejad Farsangi, Ehsan & Noori, Mohammad & Li, Shaofan, 2023. "The influence of model and measurement uncertainties on damage detection of experimental structures through recursive algorithms," Reliability Engineering and System Safety, Elsevier, vol. 239(C).
    2. Zhao, Yunjie & Cheng, Xi & Zhang, Taihong & Wang, Lei & Shao, Wei & Wiart, Joe, 2023. "A global–local attention network for uncertainty analysis of ground penetrating radar modeling," Reliability Engineering and System Safety, Elsevier, vol. 234(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhou, Taotao & Zhang, Laibin & Han, Te & Droguett, Enrique Lopez & Mosleh, Ali & Chan, Felix T.S., 2023. "An uncertainty-informed framework for trustworthy fault diagnosis in safety-critical applications," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    2. Zhao, Zeyun & Wang, Jia & Tao, Qian & Li, Andong & Chen, Yiyang, 2024. "An unknown wafer surface defect detection approach based on Incremental Learning for reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 244(C).
    3. Ma, Zhonghai & Liao, Haitao & Gao, Jianhang & Nie, Songlin & Geng, Yugang, 2023. "Physics-Informed Machine Learning for Degradation Modeling of an Electro-Hydrostatic Actuator System," Reliability Engineering and System Safety, Elsevier, vol. 229(C).
    4. Xue, Gang & Liu, Shifeng & Ren, Long & Gong, Daqing, 2024. "Risk assessment of utility tunnels through risk interaction-based deep learning," Reliability Engineering and System Safety, Elsevier, vol. 241(C).
    5. Zhou, Jie & Lin, Haifei & Li, Shugang & Jin, Hongwei & Zhao, Bo & Liu, Shihao, 2023. "Leakage diagnosis and localization of the gas extraction pipeline based on SA-PSO BP neural network," Reliability Engineering and System Safety, Elsevier, vol. 232(C).
    6. Masafumi Nakano & Akihiko Takahashi & Soichiro Takahashi, 2018. "State Space Approach to Adaptive Artificial Intelligence Modeling: Application to Financial Portfolio with Fuzzy System," CARF F-Series CARF-F-422, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    7. Izquierdo, J. & Márquez, A. Crespo & Uribetxebarria, J. & Erguido, A., 2020. "On the importance of assessing the operational context impact on maintenance management for life cycle cost of wind energy projects," Renewable Energy, Elsevier, vol. 153(C), pages 1100-1110.
    8. Li, Yi & Liu, Kailong & Foley, Aoife M. & Zülke, Alana & Berecibar, Maitane & Nanini-Maury, Elise & Van Mierlo, Joeri & Hoster, Harry E., 2019. "Data-driven health estimation and lifetime prediction of lithium-ion batteries: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 113(C), pages 1-1.
    9. Chen, Jun-Yu & Feng, Yun-Wen & Teng, Da & Lu, Cheng & Fei, Cheng-Wei, 2022. "Support vector machine-based similarity selection method for structural transient reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 223(C).
    10. Dhulipala, Somayajulu L.N. & Shields, Michael D. & Chakroborty, Promit & Jiang, Wen & Spencer, Benjamin W. & Hales, Jason D. & Labouré, Vincent M. & Prince, Zachary M. & Bolisetti, Chandrakanth & Che, 2022. "Reliability estimation of an advanced nuclear fuel using coupled active learning, multifidelity modeling, and subset simulation," Reliability Engineering and System Safety, Elsevier, vol. 226(C).
    11. Kamei, Sayaka & Taghipour, Sharareh, 2023. "A comparison study of centralized and decentralized federated learning approaches utilizing the transformer architecture for estimating remaining useful life," Reliability Engineering and System Safety, Elsevier, vol. 233(C).
    12. Wang, Jian & Gao, Shibin & Yu, Long & Liu, Xingyang & Neri, Ferrante & Zhang, Dongkai & Kou, Lei, 2024. "Uncertainty-aware trustworthy weather-driven failure risk predictor for overhead contact lines," Reliability Engineering and System Safety, Elsevier, vol. 242(C).
    13. Luo, Changqi & Zhu, Shun-Peng & Keshtegar, Behrooz & Niu, Xiaopeng & Taylan, Osman, 2023. "An enhanced uniform simulation approach coupled with SVR for efficient structural reliability analysis," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    14. Yuan, Zixia & Xiong, Guojiang & Fu, Xiaofan & Mohamed, Ali Wagdy, 2023. "Improving fault tolerance in diagnosing power system failures with optimal hierarchical extreme learning machine," Reliability Engineering and System Safety, Elsevier, vol. 236(C).
    15. Liu, Zhichen & Li, Ying & Zhang, Zhaoyi & Yu, Wenbo, 2022. "A new evacuation accessibility analysis approach based on spatial information," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    16. Zhu, Zuanyu & Cheng, Junsheng & Wang, Ping & Wang, Jian & Kang, Xin & Yang, Yu, 2023. "A novel fault diagnosis framework for rotating machinery with hierarchical multiscale symbolic diversity entropy and robust twin hyperdisk-based tensor machine," Reliability Engineering and System Safety, Elsevier, vol. 231(C).
    17. Liu, Di & Wang, Shaoping & Cui, Xiaoyu, 2022. "An artificial neural network supported Wiener process based reliability estimation method considering individual difference and measurement error," Reliability Engineering and System Safety, Elsevier, vol. 218(PB).
    18. Khakzad, Nima, 2023. "A methodology based on Dijkstra's algorithm and mathematical programming for optimal evacuation in process plants in the event of major tank fires," Reliability Engineering and System Safety, Elsevier, vol. 236(C).
    19. Zhao, Chao & Shen, Weiming, 2022. "Adaptive open set domain generalization network: Learning to diagnose unknown faults under unknown working conditions," Reliability Engineering and System Safety, Elsevier, vol. 226(C).
    20. Xie, Shuyi & Huang, Zimeng & Wu, Gang & Luo, Jinheng & Li, Lifeng & Ma, Weifeng & Wang, Bohong, 2024. "Combining precursor and Cloud Leaky noisy-OR logic gate Bayesian network for dynamic probability analysis of major accidents in the oil depots," Reliability Engineering and System Safety, Elsevier, vol. 241(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:229:y:2023:i:c:s0951832022004306. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.