IDEAS home Printed from https://ideas.repec.org/a/eee/chsofr/v167y2023ics0960077922012474.html
   My bibliography  Save this article

ReLU-type Hopfield neural network with analog hardware implementation

Author

Listed:
  • Chen, Chengjie
  • Min, Fuhong
  • Zhang, Yunzhen
  • Bao, Han

Abstract

Rectified Linear Unit (ReLU) is an extremely simple activation function with two-segment linearity and is usually used in convolutional neural network. To simplify the analog hardware implementation, this article presents a bi-neuron Hopfield neural network (HNN) using ReLU function instead of hyperbolic tangent (Tanh) function as the activation function. With the HNN model, the boundedness properties can be proved by virtue of the Lyapunov method and the stability distributions of time-varying equilibrium point are clarified according to four independent affine regions. Using numerical measures, complex dynamical behaviors are uncovered therein, including period-adding bifurcation, spiking/bursting pattern, fast-slow effect, and coexisting multistable patterns. In addition, a multiplierless electronic circuit is developed for implementing the HNN model and its experimental measurements well validate the numerical ones. The results manifest that the ReLU-type HNN model can display intricate and rich dynamics, and its analog hardware implementation is simpler than other HNN models.

Suggested Citation

  • Chen, Chengjie & Min, Fuhong & Zhang, Yunzhen & Bao, Han, 2023. "ReLU-type Hopfield neural network with analog hardware implementation," Chaos, Solitons & Fractals, Elsevier, vol. 167(C).
    • Handle: RePEc:eee:chsofr:v:167:y:2023:i:c:s0960077922012474
    DOI: 10.1016/j.chaos.2022.113068
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0960077922012474
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.chaos.2022.113068?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rech, Paulo C., 2022. "Self-excited and hidden attractors in a multistable jerk system," Chaos, Solitons & Fractals, Elsevier, vol. 164(C).
    2. Egorov, Nikita M. & Sysoev, Ilya V. & Ponomarenko, Vladimir I. & Sysoeva, Marina V., 2022. "Complex regimes in electronic neuron-like oscillators with sigmoid coupling," Chaos, Solitons & Fractals, Elsevier, vol. 160(C).
    3. Yu, Fei & Shen, Hui & Zhang, Zinan & Huang, Yuanyuan & Cai, Shuo & Du, Sichun, 2021. "Dynamics analysis, hardware implementation and engineering applications of novel multi-style attractors in a neural network under electromagnetic radiation," Chaos, Solitons & Fractals, Elsevier, vol. 152(C).
    4. Zhang, Sen & Zheng, Jiahao & Wang, Xiaoping & Zeng, Zhigang, 2021. "A novel no-equilibrium HR neuron model with hidden homogeneous extreme multistability," Chaos, Solitons & Fractals, Elsevier, vol. 145(C).
    5. Wang, Zhixiang & Zhang, Chun & Bi, Qinsheng, 2022. "Bursting oscillations with bifurcations of chaotic attractors in a modified Chua’s circuit," Chaos, Solitons & Fractals, Elsevier, vol. 165(P1).
    6. Boui A Boya, Bertrand Frederick & Ramakrishnan, Balamurali & Effa, Joseph Yves & Kengne, Jacques & Rajagopal, Karthikeyan, 2022. "The effects of symmetry breaking on the dynamics of an inertial neural system with a non-monotonic activation function: Theoretical study, asymmetric multistability and experimental investigation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 602(C).
    7. Williams-García, Rashid V. & Nicolis, Stam, 2022. "Route to chaos in a branching model of neural network dynamics," Chaos, Solitons & Fractals, Elsevier, vol. 165(P1).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chen, Xiongjian & Wang, Ning & Wang, Yiteng & Wu, Huagan & Xu, Quan, 2023. "Memristor initial-offset boosting and its bifurcation mechanism in a memristive FitzHugh-Nagumo neuron model with hidden dynamics," Chaos, Solitons & Fractals, Elsevier, vol. 174(C).
    2. Na Li & Ziyiyang Wang & Xin Lin & Haotian Sheng, 2024. "Prediction of delivery truck arrivals at container terminals: an ensemble deep learning model," Maritime Economics & Logistics, Palgrave Macmillan;International Association of Maritime Economists (IAME), vol. 26(4), pages 658-684, December.
    3. Hairong Lin & Chunhua Wang & Fei Yu & Jingru Sun & Sichun Du & Zekun Deng & Quanli Deng, 2023. "A Review of Chaotic Systems Based on Memristive Hopfield Neural Networks," Mathematics, MDPI, vol. 11(6), pages 1-18, March.
    4. Othman Abdullah Almatroud & Viet-Thanh Pham & Karthikeyan Rajagopal, 2024. "A Rectified Linear Unit-Based Memristor-Enhanced Morris–Lecar Neuron Model," Mathematics, MDPI, vol. 12(19), pages 1-10, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ding, Dawei & Chen, Xiaoyu & Yang, Zongli & Hu, Yongbing & Wang, Mouyuan & Zhang, Hongwei & Zhang, Xu, 2022. "Coexisting multiple firing behaviors of fractional-order memristor-coupled HR neuron considering synaptic crosstalk and its ARM-based implementation," Chaos, Solitons & Fractals, Elsevier, vol. 158(C).
    2. Wang, Chunhua & Luo, Dingwei & Deng, Quanli & Yang, Gang, 2024. "Dynamics analysis and FPGA implementation of discrete memristive cellular neural network with heterogeneous activation functions," Chaos, Solitons & Fractals, Elsevier, vol. 187(C).
    3. Njitacke, Zeric Tabekoueng & Ramadoss, Janarthanan & Takembo, Clovis Ntahkie & Rajagopal, Karthikeyan & Awrejcewicz, Jan, 2023. "An enhanced FitzHugh–Nagumo neuron circuit, microcontroller-based hardware implementation: Light illumination and magnetic field effects on information patterns," Chaos, Solitons & Fractals, Elsevier, vol. 167(C).
    4. Li, Fangyuan & Chen, Zhuguan & Bao, Han & Bai, Lianfa & Bao, Bocheng, 2024. "Chaos and bursting patterns in two-neuron Hopfield neural network and analog implementation," Chaos, Solitons & Fractals, Elsevier, vol. 184(C).
    5. Zhang, Sien & Yao, Wei & Xiong, Li & Wang, Yijie & Tang, Lihong & Zhang, Xin & Yu, Fei, 2024. "A Hindmarsh–Rose neuron model with electromagnetic radiation control for the mechanical optimization design," Chaos, Solitons & Fractals, Elsevier, vol. 187(C).
    6. Yan, Shaohui & Wang, Ertong & Gu, Binxian & Wang, Qiyu & Ren, Yu & Wang, Jianjian, 2022. "Analysis and finite-time synchronization of a novel double-wing chaotic system with transient chaos," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 602(C).
    7. Deng, Jie & Li, Hong-Li & Cao, Jinde & Hu, Cheng & Jiang, Haijun, 2023. "State estimation for discrete-time fractional-order neural networks with time-varying delays and uncertainties," Chaos, Solitons & Fractals, Elsevier, vol. 176(C).
    8. Singh, Jay Prakash, 2024. "A set of five generalised memristive synapses for the hidden nonlinear dynamics in three coupled neurons," Chaos, Solitons & Fractals, Elsevier, vol. 188(C).
    9. Wan, Qiuzhen & Li, Fei & Chen, Simiao & Yang, Qiao, 2023. "Symmetric multi-scroll attractors in magnetized Hopfield neural network under pulse controlled memristor and pulse current stimulation," Chaos, Solitons & Fractals, Elsevier, vol. 169(C).
    10. Ramamoorthy, Ramesh & Rajagopal, Karthikeyan & Leutcho, Gervais Dolvis & Krejcar, Ondrej & Namazi, Hamidreza & Hussain, Iqtadar, 2022. "Multistable dynamics and control of a new 4D memristive chaotic Sprott B system," Chaos, Solitons & Fractals, Elsevier, vol. 156(C).
    11. Wang, Zhixiang & Zhang, Chun & Ding, Zuqin & Bi, Qinsheng, 2023. "From period-doubling bursting to chaotic–periodic bursting in a modified Chua’s circuit," Chaos, Solitons & Fractals, Elsevier, vol. 174(C).
    12. Bao, Bocheng & Chen, Liuhui & Bao, Han & Chen, Mo & Xu, Quan, 2024. "Bifurcations to bursting oscillations in memristor-based FitzHugh-Nagumo circuit," Chaos, Solitons & Fractals, Elsevier, vol. 181(C).
    13. Fossi, Jules Tagne & Njitacke, Zeric Tabekoueng & Tankeu, William Nguimeya & Mendimi, Joseph Marie & Awrejcewicz, Jan & Atangana, Jacques, 2023. "Phase synchronization and coexisting attractors in a model of three different neurons coupled via hybrid synapses," Chaos, Solitons & Fractals, Elsevier, vol. 177(C).
    14. Cao, Hongli & Wang, Yu & Banerjee, Santo & Cao, Yinghong & Mou, Jun, 2024. "A discrete Chialvo–Rulkov neuron network coupled with a novel memristor model: Design, Dynamical analysis, DSP implementation and its application," Chaos, Solitons & Fractals, Elsevier, vol. 179(C).
    15. Chen, Xiongjian & Wang, Ning & Wang, Yiteng & Wu, Huagan & Xu, Quan, 2023. "Memristor initial-offset boosting and its bifurcation mechanism in a memristive FitzHugh-Nagumo neuron model with hidden dynamics," Chaos, Solitons & Fractals, Elsevier, vol. 174(C).
    16. Zhang, Jianlin & Bao, Han & Yu, Xihong & Chen, Bei, 2024. "Heterogeneous coexistence of extremely many attractors in adaptive synapse neuron considering memristive EMI," Chaos, Solitons & Fractals, Elsevier, vol. 178(C).
    17. Tabekoueng Njitacke, Zeric & Tsafack, Nestor & Ramakrishnan, Balamurali & Rajagopal, Kartikeyan & Kengne, Jacques & Awrejcewicz, Jan, 2021. "Complex dynamics from heterogeneous coupling and electromagnetic effect on two neurons: Application in images encryption," Chaos, Solitons & Fractals, Elsevier, vol. 153(P2).
    18. Ding, Shoukui & Wang, Ning & Bao, Han & Chen, Bei & Wu, Huagan & Xu, Quan, 2023. "Memristor synapse-coupled piecewise-linear simplified Hopfield neural network: Dynamics analysis and circuit implementation," Chaos, Solitons & Fractals, Elsevier, vol. 166(C).
    19. Zizhao Xie & Jingru Sun & Yiping Tang & Xin Tang & Oluyomi Simpson & Yichuang Sun, 2023. "A K-SVD Based Compressive Sensing Method for Visual Chaotic Image Encryption," Mathematics, MDPI, vol. 11(7), pages 1-20, March.
    20. Wang, Ning & Cui, Mengkai & Yu, Xihong & Shan, Yufan & Xu, Quan, 2023. "Generating multi-folded hidden Chua’s attractors: Two-case study," Chaos, Solitons & Fractals, Elsevier, vol. 177(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:chsofr:v:167:y:2023:i:c:s0960077922012474. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Thayer, Thomas R. (email available below). General contact details of provider: https://www.journals.elsevier.com/chaos-solitons-and-fractals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.