IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-46586-0.html
   My bibliography  Save this article

Local prediction-learning in high-dimensional spaces enables neural networks to plan

Author

Listed:
  • Christoph Stöckl

    (Graz University of Technology)

  • Yukun Yang

    (Graz University of Technology)

  • Wolfgang Maass

    (Graz University of Technology)

Abstract

Planning and problem solving are cornerstones of higher brain function. But we do not know how the brain does that. We show that learning of a suitable cognitive map of the problem space suffices. Furthermore, this can be reduced to learning to predict the next observation through local synaptic plasticity. Importantly, the resulting cognitive map encodes relations between actions and observations, and its emergent high-dimensional geometry provides a sense of direction for reaching distant goals. This quasi-Euclidean sense of direction provides a simple heuristic for online planning that works almost as well as the best offline planning algorithms from AI. If the problem space is a physical space, this method automatically extracts structural regularities from the sequence of observations that it receives so that it can generalize to unseen parts. This speeds up learning of navigation in 2D mazes and the locomotion with complex actuator systems, such as legged bodies. The cognitive map learner that we propose does not require a teacher, similar to self-attention networks (Transformers). But in contrast to Transformers, it does not require backpropagation of errors or very large datasets for learning. Hence it provides a blue-print for future energy-efficient neuromorphic hardware that acquires advanced cognitive capabilities through autonomous on-chip learning.

Suggested Citation

  • Christoph Stöckl & Yukun Yang & Wolfgang Maass, 2024. "Local prediction-learning in high-dimensional spaces enables neural networks to plan," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-46586-0
    DOI: 10.1038/s41467-024-46586-0
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-46586-0
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-46586-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jing Pei & Lei Deng & Sen Song & Mingguo Zhao & Youhui Zhang & Shuang Wu & Guanrui Wang & Zhe Zou & Zhenzhi Wu & Wei He & Feng Chen & Ning Deng & Si Wu & Yu Wang & Yujie Wu & Zheyu Yang & Cheng Ma & G, 2019. "Towards artificial general intelligence with hybrid Tianjic chip architecture," Nature, Nature, vol. 572(7767), pages 106-111, August.
    2. Mingyi Rao & Hao Tang & Jiangbin Wu & Wenhao Song & Max Zhang & Wenbo Yin & Ye Zhuo & Fatemeh Kiani & Benjamin Chen & Xiangqi Jiang & Hefei Liu & Hung-Yu Chen & Rivu Midya & Fan Ye & Hao Jiang & Zhong, 2023. "Thousands of conductance levels in memristors integrated on CMOS," Nature, Nature, vol. 615(7954), pages 823-829, March.
    3. Tyler D. Marks & Michael J. Goard, 2021. "Author Correction: Stimulus-dependent representational drift in primary visual cortex," Nature Communications, Nature, vol. 12(1), pages 1-1, December.
    4. Payam Piray & Nathaniel D. Daw, 2021. "Linear reinforcement learning in planning, grid fields, and cognitive control," Nature Communications, Nature, vol. 12(1), pages 1-20, December.
    5. Tyler D. Marks & Michael J. Goard, 2021. "Stimulus-dependent representational drift in primary visual cortex," Nature Communications, Nature, vol. 12(1), pages 1-16, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Han Chin Wang & Amy M. LeMessurier & Daniel E. Feldman, 2022. "Tuning instability of non-columnar neurons in the salt-and-pepper whisker map in somatosensory cortex," Nature Communications, Nature, vol. 13(1), pages 1-18, December.
    2. Luis M. Franco & Michael J. Goard, 2024. "Differential stability of task variable representations in retrosplenial cortex," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    3. Hannah Muysers & Hung-Ling Chen & Johannes Hahn & Shani Folschweiller & Torfi Sigurdsson & Jonas-Frederic Sauer & Marlene Bartos, 2024. "A persistent prefrontal reference frame across time and task rules," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    4. Ravi Pancholi & Lauren Ryan & Simon Peron, 2023. "Learning in a sensory cortical microstimulation task is associated with elevated representational stability," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    5. Percy K. Mistry & Anthony Strock & Ruizhe Liu & Griffin Young & Vinod Menon, 2023. "Learning-induced reorganization of number neurons and emergence of numerical representations in a biologically inspired neural network," Nature Communications, Nature, vol. 14(1), pages 1-21, December.
    6. Zvi N. Roth & Elisha P. Merriam, 2023. "Representations in human primary visual cortex drift over time," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    7. Wang, Huan & Li, Yan-Fu & Zhang, Ying, 2023. "Bioinspired spiking spatiotemporal attention framework for lithium-ion batteries state-of-health estimation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 188(C).
    8. Jongmin Lee & Bum Ho Jeong & Eswaran Kamaraj & Dohyung Kim & Hakjun Kim & Sanghyuk Park & Hui Joon Park, 2023. "Light-enhanced molecular polarity enabling multispectral color-cognitive memristor for neuromorphic visual system," Nature Communications, Nature, vol. 14(1), pages 1-19, December.
    9. Zhiyuan Li & Zhongshao Li & Wei Tang & Jiaping Yao & Zhipeng Dou & Junjie Gong & Yongfei Li & Beining Zhang & Yunxiao Dong & Jian Xia & Lin Sun & Peng Jiang & Xun Cao & Rui Yang & Xiangshui Miao & Ron, 2024. "Crossmodal sensory neurons based on high-performance flexible memristors for human-machine in-sensor computing system," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    10. Man Yao & Ole Richter & Guangshe Zhao & Ning Qiao & Yannan Xing & Dingheng Wang & Tianxiang Hu & Wei Fang & Tugba Demirci & Michele Marchi & Lei Deng & Tianyi Yan & Carsten Nielsen & Sadique Sheik & C, 2024. "Spike-based dynamic computing with asynchronous sensing-computing neuromorphic chip," Nature Communications, Nature, vol. 15(1), pages 1-18, December.
    11. Wang, Huan & Li, Yan-Fu, 2023. "Bioinspired membrane learnable spiking neural network for autonomous vehicle sensors fault diagnosis under open environments," Reliability Engineering and System Safety, Elsevier, vol. 233(C).
    12. Peng Chen & Fenghao Liu & Peng Lin & Peihong Li & Yu Xiao & Bihua Zhang & Gang Pan, 2023. "Open-loop analog programmable electrochemical memory array," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    13. Yulin Feng & Yizhou Zhang & Zheng Zhou & Peng Huang & Lifeng Liu & Xiaoyan Liu & Jinfeng Kang, 2024. "Memristor-based storage system with convolutional autoencoder-based image compression network," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    14. Zhang, Xu & Min, Fuhong & Dou, Yiping & Xu, Yeyin, 2023. "Bifurcation analysis of a modified FitzHugh-Nagumo neuron with electric field," Chaos, Solitons & Fractals, Elsevier, vol. 170(C).
    15. Chenhao Wang & Xinyi Xu & Xiaodong Pi & Mark D. Butala & Wen Huang & Lei Yin & Wenbing Peng & Munir Ali & Srikrishna Chanakya Bodepudi & Xvsheng Qiao & Yang Xu & Wei Sun & Deren Yang, 2022. "Neuromorphic device based on silicon nanosheets," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    16. Jaeseoung Park & Ashwani Kumar & Yucheng Zhou & Sangheon Oh & Jeong-Hoon Kim & Yuhan Shi & Soumil Jain & Gopabandhu Hota & Erbin Qiu & Amelie L. Nagle & Ivan K. Schuller & Catherine D. Schuman & Gert , 2024. "Multi-level, forming and filament free, bulk switching trilayer RRAM for neuromorphic computing at the edge," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    17. Mingrui Jiang & Keyi Shan & Chengping He & Can Li, 2023. "Efficient combinatorial optimization by quantum-inspired parallel annealing in analogue memristor crossbar," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    18. Pietro Belleri & Judith Pons i Tarrés & Iain McCulloch & Paul W. M. Blom & Zsolt M. Kovács-Vajna & Paschalis Gkoupidenis & Fabrizio Torricelli, 2024. "Unravelling the operation of organic artificial neurons for neuromorphic bioelectronics," Nature Communications, Nature, vol. 15(1), pages 1-16, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-46586-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.