IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-50633-1.html
   My bibliography  Save this article

Neuromorphic overparameterisation and few-shot learning in multilayer physical neural networks

Author

Listed:
  • Kilian D. Stenning

    (Imperial College London
    Imperial College London)

  • Jack C. Gartside

    (Imperial College London
    Imperial College London)

  • Luca Manneschi

    (University of Sheffield)

  • Christopher T. S. Cheung

    (Imperial College London)

  • Tony Chen

    (Imperial College London)

  • Alex Vanstone

    (Imperial College London)

  • Jake Love

    (University of Duisburg-Essen)

  • Holly Holder

    (Imperial College London)

  • Francesco Caravelli

    (Los Alamos National Laboratory)

  • Hidekazu Kurebayashi

    (University College London
    University College London
    Tohoku University)

  • Karin Everschor-Sitte

    (University of Duisburg-Essen)

  • Eleni Vasilaki

    (University of Sheffield)

  • Will R. Branford

    (Imperial College London
    Imperial College London)

Abstract

Physical neuromorphic computing, exploiting the complex dynamics of physical systems, has seen rapid advancements in sophistication and performance. Physical reservoir computing, a subset of neuromorphic computing, faces limitations due to its reliance on single systems. This constrains output dimensionality and dynamic range, limiting performance to a narrow range of tasks. Here, we engineer a suite of nanomagnetic array physical reservoirs and interconnect them in parallel and series to create a multilayer neural network architecture. The output of one reservoir is recorded, scaled and virtually fed as input to the next reservoir. This networked approach increases output dimensionality, internal dynamics and computational performance. We demonstrate that a physical neuromorphic system can achieve an overparameterised state, facilitating meta-learning on small training sets and yielding strong performance across a wide range of tasks. Our approach’s efficacy is further demonstrated through few-shot learning, where the system rapidly adapts to new tasks.

Suggested Citation

  • Kilian D. Stenning & Jack C. Gartside & Luca Manneschi & Christopher T. S. Cheung & Tony Chen & Alex Vanstone & Jake Love & Holly Holder & Francesco Caravelli & Hidekazu Kurebayashi & Karin Everschor-, 2024. "Neuromorphic overparameterisation and few-shot learning in multilayer physical neural networks," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-50633-1
    DOI: 10.1038/s41467-024-50633-1
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-50633-1
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-50633-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Alice Mizrahi & Tifenn Hirtzlin & Akio Fukushima & Hitoshi Kubota & Shinji Yuasa & Julie Grollier & Damien Querlioz, 2018. "Neural-like computing with populations of superparamagnetic basis functions," Nature Communications, Nature, vol. 9(1), pages 1-11, December.
    2. Logan G. Wright & Tatsuhiro Onodera & Martin M. Stein & Tianyu Wang & Darren T. Schachter & Zoey Hu & Peter L. McMahon, 2022. "Deep physical neural networks trained with backpropagation," Nature, Nature, vol. 601(7894), pages 549-555, January.
    3. Xiangpeng Liang & Yanan Zhong & Jianshi Tang & Zhengwu Liu & Peng Yao & Keyang Sun & Qingtian Zhang & Bin Gao & Hadi Heidari & He Qian & Huaqiang Wu, 2022. "Rotating neurons for all-analog implementation of cyclic reservoir computing," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    4. Wenjie Hu & Zefeng Zhang & Yanghui Liao & Qiang Li & Yang Shi & Huanyu Zhang & Xumeng Zhang & Chang Niu & Yu Wu & Weichao Yu & Xiaodong Zhou & Hangwen Guo & Wenbin Wang & Jiang Xiao & Lifeng Yin & Qi , 2023. "Distinguishing artificial spin ice states using magnetoresistance effect for neuromorphic computing," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    5. Fadi Jebali & Atreya Majumdar & Clément Turck & Kamel-Eddine Harabi & Mathieu-Coumba Faye & Eloi Muhr & Jean-Pierre Walder & Oleksandr Bilousov & Amadéo Michaud & Elisa Vianello & Tifenn Hirtzlin & Fr, 2024. "Powering AI at the edge: A robust, memristor-based binarized neural network with near-memory computing and miniaturized solar cell," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    6. Shaikh, Faisal Karim & Zeadally, Sherali, 2016. "Energy harvesting in wireless sensor networks: A comprehensive review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 55(C), pages 1041-1054.
    7. Ádám Papp & Wolfgang Porod & Gyorgy Csaba, 2021. "Nanoscale neural network using non-linear spin-wave interference," Nature Communications, Nature, vol. 12(1), pages 1-8, December.
    8. Jacob Torrejon & Mathieu Riou & Flavio Abreu Araujo & Sumito Tsunegi & Guru Khalsa & Damien Querlioz & Paolo Bortolotti & Vincent Cros & Kay Yakushiji & Akio Fukushima & Hitoshi Kubota & Shinji Yuasa , 2017. "Neuromorphic computing with nanoscale spintronic oscillators," Nature, Nature, vol. 547(7664), pages 428-431, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhiwei Chen & Wenjie Li & Zhen Fan & Shuai Dong & Yihong Chen & Minghui Qin & Min Zeng & Xubing Lu & Guofu Zhou & Xingsen Gao & Jun-Ming Liu, 2023. "All-ferroelectric implementation of reservoir computing," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    2. Xing Chen & Flavio Abreu Araujo & Mathieu Riou & Jacob Torrejon & Dafiné Ravelosona & Wang Kang & Weisheng Zhao & Julie Grollier & Damien Querlioz, 2022. "Forecasting the outcome of spintronic experiments with Neural Ordinary Differential Equations," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    3. Jérémie Laydevant & Danijela Marković & Julie Grollier, 2024. "Training an Ising machine with equilibrium propagation," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    4. Liao, Zhiqiang & Ma, Kaijie & Tang, Siyi & Sarker, Md Shamim & Yamahara, Hiroyasu & Tabata, Hitoshi, 2021. "Phase locking of ultra-low power consumption stochastic magnetic bits induced by colored noise," Chaos, Solitons & Fractals, Elsevier, vol. 151(C).
    5. Mitsumasa Nakajima & Katsuma Inoue & Kenji Tanaka & Yasuo Kuniyoshi & Toshikazu Hashimoto & Kohei Nakajima, 2022. "Physical deep learning with biologically inspired training method: gradient-free approach for physical hardware," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    6. Lukas Körber & Christopher Heins & Tobias Hula & Joo-Von Kim & Sonia Thlang & Helmut Schultheiss & Jürgen Fassbender & Katrin Schultheiss, 2023. "Pattern recognition in reciprocal space with a magnon-scattering reservoir," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    7. Dimitrios A. Papathanasopoulos & Konstantinos N. Giannousakis & Evangelos S. Dermatas & Epaminondas D. Mitronikas, 2021. "Vibration Monitoring for Position Sensor Fault Diagnosis in Brushless DC Motor Drives," Energies, MDPI, vol. 14(8), pages 1-24, April.
    8. Oleksii M. Volkov & Oleksandr V. Pylypovskyi & Fabrizio Porrati & Florian Kronast & Jose A. Fernandez-Roldan & Attila Kákay & Alexander Kuprava & Sven Barth & Filipp N. Rybakov & Olle Eriksson & Sebas, 2024. "Three-dimensional magnetic nanotextures with high-order vorticity in soft magnetic wireframes," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    9. Fan Cai & Yuesong Jiang & Wanqing Song & Kai-Hung Lu & Tongbo Zhu, 2024. "Short-Term Wind Turbine Blade Icing Wind Power Prediction Based on PCA-fLsm," Energies, MDPI, vol. 17(6), pages 1-15, March.
    10. Farnaz Derakhshan & Shamim Yousefi, 2019. "A review on the applications of multiagent systems in wireless sensor networks," International Journal of Distributed Sensor Networks, , vol. 15(5), pages 15501477198, May.
    11. Ashraf Virk, Mati-ur-Rasool & Mysorewala, Muhammad Faizan & Cheded, Lahouari & Aliyu, AbdulRahman, 2022. "Review of energy harvesting techniques in wireless sensor-based pipeline monitoring networks," Renewable and Sustainable Energy Reviews, Elsevier, vol. 157(C).
    12. Xiangpeng Liang & Yanan Zhong & Jianshi Tang & Zhengwu Liu & Peng Yao & Keyang Sun & Qingtian Zhang & Bin Gao & Hadi Heidari & He Qian & Huaqiang Wu, 2022. "Rotating neurons for all-analog implementation of cyclic reservoir computing," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    13. Yang, Chen & Xue, RuiPu & Li, Xu & Zhang, XiaoQing & Wu, ZhenYu, 2020. "Power performance of solar energy harvesting system under typical indoor light sources," Renewable Energy, Elsevier, vol. 161(C), pages 836-845.
    14. Abbas, Khizar & Han, Mengyao & Xu, Deyi & Butt, Khalid Manzoor & Baz, Khan & Cheng, Jinhua & Zhu, Yongguang & Hussain, Sanwal, 2024. "Exploring synergistic and individual causal effects of rare earth elements and renewable energy on multidimensional economic complexity for sustainable economic development," Applied Energy, Elsevier, vol. 364(C).
    15. Klaus Raab & Maarten A. Brems & Grischa Beneke & Takaaki Dohi & Jan Rothörl & Fabian Kammerbauer & Johan H. Mentink & Mathias Kläui, 2022. "Brownian reservoir computing realized using geometrically confined skyrmion dynamics," Nature Communications, Nature, vol. 13(1), pages 1-6, December.
    16. Davide Girardi & Simone Finizio & Claire Donnelly & Guglielmo Rubini & Sina Mayr & Valerio Levati & Simone Cuccurullo & Federico Maspero & Jörg Raabe & Daniela Petti & Edoardo Albisetti, 2024. "Three-dimensional spin-wave dynamics, localization and interference in a synthetic antiferromagnet," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    17. Seou Choi & Yannick Salamin & Charles Roques-Carmes & Rumen Dangovski & Di Luo & Zhuo Chen & Michael Horodynski & Jamison Sloan & Shiekh Zia Uddin & Marin Soljačić, 2024. "Photonic probabilistic machine learning using quantum vacuum noise," Nature Communications, Nature, vol. 15(1), pages 1-8, December.
    18. Wang, Yilong & Yang, Zhengbao & Cao, Dengqing, 2021. "On the offset distance of rotational piezoelectric energy harvesters," Energy, Elsevier, vol. 220(C).
    19. Long Liu & Di Wang & Dandan Wang & Yan Sun & Huai Lin & Xiliang Gong & Yifan Zhang & Ruifeng Tang & Zhihong Mai & Zhipeng Hou & Yumeng Yang & Peng Li & Lan Wang & Qing Luo & Ling Li & Guozhong Xing & , 2024. "Domain wall magnetic tunnel junction-based artificial synapses and neurons for all-spin neuromorphic hardware," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    20. Cao, Dong-Xing & Lu, Yi-Ming & Lai, Siu-Kai & Mao, Jia-Jia & Guo, Xiang-Ying & Shen, Yong-Jun, 2022. "A novel soft encapsulated multi-directional and multi-modal piezoelectric vibration energy harvester," Energy, Elsevier, vol. 254(PB).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-50633-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.