IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1005669.html
   My bibliography  Save this article

Learning and executing goal-directed choices by internally generated sequences in spiking neural circuits

Author

Listed:
  • John Palmer
  • Adam Keane
  • Pulin Gong

Abstract

Recent neural ensemble recordings have established a link between goal-directed spatial decision making and internally generated neural sequences in the hippocampus of rats. To elucidate the synaptic mechanisms of these sequences underlying spatial decision making processes, we develop and investigate a spiking neural circuit model endowed with a combination of two synaptic plasticity mechanisms including spike-timing dependent plasticity (STDP) and synaptic scaling. In this model, the interplay of the combined synaptic plasticity mechanisms and network dynamics gives rise to neural sequences which propagate ahead of the animals’ decision point to reach goal locations. The dynamical properties of these forward-sweeping sequences and the rates of correct binary choices executed by these sequences are quantitatively consistent with experimental observations; this consistency, however, is lost in our model when only one of STDP or synaptic scaling is included. We further demonstrate that such sequence-based decision making in our network model can adaptively respond to time-varying and probabilistic associations of cues and goal locations, and that our model performs as well as an optimal Kalman filter model. Our results thus suggest that the combination of plasticity phenomena on different timescales provides a candidate mechanism for forming internally generated neural sequences and for implementing adaptive spatial decision making.Author summary: Adaptive goal-directed decision making is critical for animals, robots and humans to navigate through space. In this study, we propose a novel neural mechanism for implementing spatial decision making in cued-choice tasks. We show that in a spiking neural circuit model, the interplay of network dynamics and a combination of two synaptic plasticity rules, STDP and synaptic scaling, gives rise to neural sequences. When a model rat pauses around a decision point, these sequences propagate ahead of the animal’s current location and travel towards a goal location. The dynamical properties of these forward-sweeping sequences and the rate of correct responses made by them are consistent with experimental data. In addition, we demonstrate that STDP when complemented by slower synaptic scaling enables neural sequences to make adaptive choices under probabilistic and time-varying cue-goal associations. The adaptive performance of our sequence-based network is comparable to a mathematical model, namely the Kalman filter, which is optimal for this adaptive task. Our results thus shed new light on our understanding of neural mechanisms underlying goal-directed decision making.

Suggested Citation

  • John Palmer & Adam Keane & Pulin Gong, 2017. "Learning and executing goal-directed choices by internally generated sequences in spiking neural circuits," PLOS Computational Biology, Public Library of Science, vol. 13(7), pages 1-23, July.
  • Handle: RePEc:plo:pcbi00:1005669
    DOI: 10.1371/journal.pcbi.1005669
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005669
    Download Restriction: no

    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1005669&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pcbi.1005669?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Richard H. R. Hahnloser & Alexay A. Kozhevnikov & Michale S. Fee, 2002. "An ultra-sparse code underliesthe generation of neural sequences in a songbird," Nature, Nature, vol. 419(6902), pages 65-70, September.
    2. Brad E. Pfeiffer & David J. Foster, 2013. "Hippocampal place-cell sequences depict future paths to remembered goals," Nature, Nature, vol. 497(7447), pages 74-79, May.
    3. Gina G. Turrigiano & Kenneth R. Leslie & Niraj S. Desai & Lana C. Rutherford & Sacha B. Nelson, 1998. "Activity-dependent scaling of quantal amplitude in neocortical neurons," Nature, Nature, vol. 391(6670), pages 892-896, February.
    4. Louis-Emmanuel Martinet & Denis Sheynikhovich & Karim Benchenane & Angelo Arleo, 2011. "Spatial Learning and Action Planning in a Prefrontal Cortical Network Model," PLOS Computational Biology, Public Library of Science, vol. 7(5), pages 1-21, May.
    5. Li I. Zhang & Huizhong W. Tao & Christine E. Holt & William A. Harris & Mu-ming Poo, 1998. "A critical window for cooperation and competition among developing retinotectal synapses," Nature, Nature, vol. 395(6697), pages 37-44, September.
    6. Christopher D. Harvey & Philip Coen & David W. Tank, 2012. "Choice-specific sequences in parietal cortex during a virtual-navigation decision task," Nature, Nature, vol. 484(7392), pages 62-68, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hong Yu & Xinkuan Xiang & Zongming Chen & Xu Wang & Jiaqi Dai & Xinxin Wang & Pengcheng Huang & Zheng-dong Zhao & Wei L. Shen & Haohong Li, 2021. "Periaqueductal gray neurons encode the sequential motor program in hunting behavior of mice," Nature Communications, Nature, vol. 12(1), pages 1-15, December.
    2. David Kappel & Bernhard Nessler & Wolfgang Maass, 2014. "STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning," PLOS Computational Biology, Public Library of Science, vol. 10(3), pages 1-22, March.
    3. J Matthew Mahoney & Ali S Titiz & Amanda E Hernan & Rod C Scott, 2016. "Short-Range Temporal Interactions in Sleep; Hippocampal Spike Avalanches Support a Large Milieu of Sequential Activity Including Replay," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-25, February.
    4. Marta Huelin Gorriz & Masahiro Takigawa & Daniel Bendor, 2023. "The role of experience in prioritizing hippocampal replay," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    5. Matteo Saponati & Martin Vinck, 2023. "Sequence anticipation and spike-timing-dependent plasticity emerge from a predictive learning rule," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    6. Nicolas Cazin & Martin Llofriu Alonso & Pablo Scleidorovich Chiodi & Tatiana Pelc & Bruce Harland & Alfredo Weitzenfeld & Jean-Marc Fellous & Peter Ford Dominey, 2019. "Reservoir computing model of prefrontal cortex creates novel combinations of previous navigation sequences from hippocampal place-cell replay with spatial reward propagation," PLOS Computational Biology, Public Library of Science, vol. 15(7), pages 1-32, July.
    7. Niranjan Chakravarthy & Shivkumar Sabesan & Kostas Tsakalis & Leon Iasemidis, 2009. "Controlling epileptic seizures in a neural mass model," Journal of Combinatorial Optimization, Springer, vol. 17(1), pages 98-116, January.
    8. Damien M O’Halloran, 2020. "Simulation model of CA1 pyramidal neurons reveal opposing roles for the Na+/Ca2+ exchange current and Ca2+-activated K+ current during spike-timing dependent synaptic plasticity," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-12, March.
    9. Yu, Haitao & Guo, Xinmeng & Wang, Jiang & Deng, Bin & Wei, Xile, 2015. "Spike coherence and synchronization on Newman–Watts small-world neuronal networks modulated by spike-timing-dependent plasticity," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 419(C), pages 307-317.
    10. Sacha Jennifer van Albada & Moritz Helias & Markus Diesmann, 2015. "Scalability of Asynchronous Networks Is Limited by One-to-One Mapping between Effective Connectivity and Correlations," PLOS Computational Biology, Public Library of Science, vol. 11(9), pages 1-37, September.
    11. Christian Keck & Cristina Savin & Jörg Lücke, 2012. "Feedforward Inhibition and Synaptic Scaling – Two Sides of the Same Coin?," PLOS Computational Biology, Public Library of Science, vol. 8(3), pages 1-15, March.
    12. Iris Reuveni & Sourav Ghosh & Edi Barkai, 2017. "Real Time Multiplicative Memory Amplification Mediated by Whole-Cell Scaling of Synaptic Response in Key Neurons," PLOS Computational Biology, Public Library of Science, vol. 13(1), pages 1-31, January.
    13. Asako Noguchi & Roman Huszár & Shota Morikawa & György Buzsáki & Yuji Ikegaya, 2022. "Inhibition allocates spikes during hippocampal ripples," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    14. Zhewei Zhang & Yuji K. Takahashi & Marlian Montesinos-Cartegena & Thorsten Kahnt & Angela J. Langdon & Geoffrey Schoenbaum, 2024. "Expectancy-related changes in firing of dopamine neurons depend on hippocampus," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    15. Benjamin R Cowley & Matthew A Smith & Adam Kohn & Byron M Yu, 2016. "Stimulus-Driven Population Activity Patterns in Macaque Primary Visual Cortex," PLOS Computational Biology, Public Library of Science, vol. 12(12), pages 1-31, December.
    16. Anli A. Liu & Simon Henin & Saman Abbaspoor & Anatol Bragin & Elizabeth A. Buffalo & Jordan S. Farrell & David J. Foster & Loren M. Frank & Tamara Gedankien & Jean Gotman & Jennifer A. Guidera & Kari , 2022. "A consensus statement on detection of hippocampal sharp wave ripples and differentiation from other fast oscillations," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    17. A. Barri & M. T. Wiechert & M. Jazayeri & D. A. DiGregorio, 2022. "Synaptic basis of a sub-second representation of time in a neural circuit model," Nature Communications, Nature, vol. 13(1), pages 1-18, December.
    18. Trygve Solstad & Hosam N Yousif & Terrence J Sejnowski, 2014. "Place Cell Rate Remapping by CA3 Recurrent Collaterals," PLOS Computational Biology, Public Library of Science, vol. 10(6), pages 1-10, June.
    19. Francesco Donnarumma & Domenico Maisto & Giovanni Pezzulo, 2016. "Problem Solving as Probabilistic Inference with Subgoaling: Explaining Human Successes and Pitfalls in the Tower of Hanoi," PLOS Computational Biology, Public Library of Science, vol. 12(4), pages 1-30, April.
    20. Mizusaki, Beatriz E.P. & Agnes, Everton J. & Erichsen, Rubem & Brunnet, Leonardo G., 2017. "Learning and retrieval behavior in recurrent neural networks with pre-synaptic dependent homeostatic plasticity," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 479(C), pages 279-286.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1005669. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.