IDEAS home Printed from https://ideas.repec.org/a/eee/chsofr/v162y2022ics0960077922006397.html
   My bibliography  Save this article

Adaptive stochastic resonance based convolutional neural network for image classification

Author

Listed:
  • Duan, Lingling
  • Ren, Yuhao
  • Duan, Fabing

Abstract

In this paper, we exploit the adaptive stochastic resonance effect in the convolutional neural network with threshold activation functions for enabling the back-propagation gradient computation. During training, the injection of noise into the threshold activation function makes it to be differentiable at forward pass, thus exact gradients of the loss function with respect to network parameters including the injected noise level can smoothly propagate at backforward pass. This training method effectively uses the non-zero noise to improve the performance of the designed convolutional neural network. In the testing phase, the differentiable activation function is decoupled into a set of threshold functions driven by mutually independent noise components, which endows a hardware-friendly feature of the designed neural network. Experiments show that the designed low-precision convolutional neural network has a comparable performance with the full-precision one with ReLU activation functions on two benchmark data sets. These results also extend the potential application of adaptive stochastic resonance into the convolutional neural network for image classification.

Suggested Citation

  • Duan, Lingling & Ren, Yuhao & Duan, Fabing, 2022. "Adaptive stochastic resonance based convolutional neural network for image classification," Chaos, Solitons & Fractals, Elsevier, vol. 162(C).
  • Handle: RePEc:eee:chsofr:v:162:y:2022:i:c:s0960077922006397
    DOI: 10.1016/j.chaos.2022.112429
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960077922006397
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.chaos.2022.112429?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dong, Haitao & Shen, Xiaohong & He, Ke & Wang, Haiyan, 2020. "Nonlinear filtering effects of intrawell matched stochastic resonance with barrier constrainted duffing system for ship radiated line signature extraction," Chaos, Solitons & Fractals, Elsevier, vol. 141(C).
    2. Li, Mengdi & Shi, Peiming & Zhang, Wenyue & Han, Dongying, 2021. "A novel underdamped continuous unsaturation bistable stochastic resonance method and its application," Chaos, Solitons & Fractals, Elsevier, vol. 151(C).
    3. Yu, Dong & Lu, Lulu & Wang, Guowei & Yang, Lijian & Jia, Ya, 2021. "Synchronization mode transition induced by bounded noise in multiple time-delays coupled FitzHugh–Nagumo model," Chaos, Solitons & Fractals, Elsevier, vol. 147(C).
    4. Liu, Jian & Qiao, Zijian & Ding, Xiaojian & Hu, Bing & Zang, Chuanlai, 2021. "Stochastic resonance induced weak signal enhancement over controllable potential-well asymmetry," Chaos, Solitons & Fractals, Elsevier, vol. 146(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Duan, Fabing & Chapeau-Blondeau, François & Abbott, Derek, 2024. "Optimized injection of noise in activation functions to improve generalization of neural networks," Chaos, Solitons & Fractals, Elsevier, vol. 178(C).
    2. Suo, Jian & Wang, Haiyan & Lian, Wei & Dong, Haitao & Shen, Xiaohong & Yan, Yongsheng, 2023. "Feed-forward cascaded stochastic resonance and its application in ship radiated line signature extraction," Chaos, Solitons & Fractals, Elsevier, vol. 174(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ren, Yuhao & Pan, Yan & Duan, Fabing, 2022. "SNR gain enhancement in a generalized matched filter using artificial optimal noise," Chaos, Solitons & Fractals, Elsevier, vol. 155(C).
    2. Yang, GuiJiang & Ai, Hao & Liu, Wei & Wang, Qiubao, 2023. "Weak signal detection based on variable-situation-potential with time-delay feedback and colored noise," Chaos, Solitons & Fractals, Elsevier, vol. 169(C).
    3. Yu, Dong & Wang, Guowei & Ding, Qianming & Li, Tianyu & Jia, Ya, 2022. "Effects of bounded noise and time delay on signal transmission in excitable neural networks," Chaos, Solitons & Fractals, Elsevier, vol. 157(C).
    4. Liu, Jian & Qiao, Zijian & Ding, Xiaojian & Hu, Bing & Zang, Chuanlai, 2021. "Stochastic resonance induced weak signal enhancement over controllable potential-well asymmetry," Chaos, Solitons & Fractals, Elsevier, vol. 146(C).
    5. Ai, Hao & Yang, GuiJiang & Liu, Wei & Wang, Qiubao, 2023. "A fast search method for optimal parameters of stochastic resonance based on stochastic bifurcation and its application in fault diagnosis of rolling bearings," Chaos, Solitons & Fractals, Elsevier, vol. 168(C).
    6. Liao, Zhiqiang & Ma, Kaijie & Tang, Siyi & Sarker, Md Shamim & Yamahara, Hiroyasu & Tabata, Hitoshi, 2021. "Phase locking of ultra-low power consumption stochastic magnetic bits induced by colored noise," Chaos, Solitons & Fractals, Elsevier, vol. 151(C).
    7. Jiang, Jiahao & Li, Kaiyuan & Guo, Wei & Du, Luchun, 2021. "Energetic and entropic vibrational resonance," Chaos, Solitons & Fractals, Elsevier, vol. 152(C).
    8. Cui, Yingxue & Ning, Lijuan, 2023. "Transport of coupled particles in fractional feedback ratchet driven by Bounded noise," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 615(C).
    9. Fu, Peng & Wang, Can-Jun & Yang, Ke-Li & Li, Xu-Bo & Yu, Biao, 2022. "Reentrance-like vibrational resonance in a fractional-order birhythmic biological system," Chaos, Solitons & Fractals, Elsevier, vol. 155(C).
    10. Shadizadeh, S. Mohadeseh & Nazarimehr, Fahimeh & Jafari, Sajad & Rajagopal, Karthikeyan, 2022. "Investigating different synaptic connections of the Chay neuron model," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 607(C).
    11. Wang, Guowei & Wu, Yong & Xiao, Fangli & Ye, Zhiqiu & Jia, Ya, 2022. "Non-Gaussian noise and autapse-induced inverse stochastic resonance in bistable Izhikevich neural system under electromagnetic induction," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 598(C).
    12. Yu, Xingwang & Ma, Yuanlin, 2022. "Steady-state analysis of the stochastic Beverton-Holt growth model driven by correlated colored noises," Chaos, Solitons & Fractals, Elsevier, vol. 158(C).
    13. Yu, Dong & Wu, Yong & Yang, Lijian & Zhao, Yunjie & Jia, Ya, 2023. "Effect of topology on delay-induced multiple resonances in locally driven systems," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 609(C).
    14. Hu, Dongpo & Ma, Linyi & Song, Zigen & Zheng, Zhaowen & Cheng, Lifang & Liu, Ming, 2024. "Multiple bifurcations of a time-delayed coupled FitzHugh–Rinzel neuron system with chemical and electrical couplings," Chaos, Solitons & Fractals, Elsevier, vol. 180(C).
    15. Li, Mengdi & Shi, Peiming & Zhang, Wenyue & Han, Dongying, 2021. "A novel underdamped continuous unsaturation bistable stochastic resonance method and its application," Chaos, Solitons & Fractals, Elsevier, vol. 151(C).
    16. Liu, Huixia & Lu, Lulu & Zhu, Yuan & Wei, Zhouchao & Yi, Ming, 2022. "Stochastic resonance: The response to envelope modulation signal for neural networks with different topologies," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 607(C).
    17. Hu, Xueyan & Ding, Qianming & Wu, Yong & Huang, Weifang & Yang, Lijian & Jia, Ya, 2024. "Dynamical rewiring promotes synchronization in memristive FitzHugh-Nagumo neuronal networks," Chaos, Solitons & Fractals, Elsevier, vol. 184(C).
    18. Suo, Jian & Wang, Haiyan & Lian, Wei & Dong, Haitao & Shen, Xiaohong & Yan, Yongsheng, 2023. "Feed-forward cascaded stochastic resonance and its application in ship radiated line signature extraction," Chaos, Solitons & Fractals, Elsevier, vol. 174(C).
    19. Wang, Guowei & Yu, Dong & Ding, Qianming & Li, Tianyu & Jia, Ya, 2021. "Effects of electric field on multiple vibrational resonances in Hindmarsh-Rose neuronal systems," Chaos, Solitons & Fractals, Elsevier, vol. 150(C).
    20. Li, Tianyu & Wu, Yong & Yang, Lijian & Zhan, Xuan & Jia, Ya, 2022. "Spike-timing-dependent plasticity enhances chaotic resonance in small-world network," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 606(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:chsofr:v:162:y:2022:i:c:s0960077922006397. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Thayer, Thomas R. (email available below). General contact details of provider: https://www.journals.elsevier.com/chaos-solitons-and-fractals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.