IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1006624.html
   My bibliography  Save this article

Reservoir computing model of prefrontal cortex creates novel combinations of previous navigation sequences from hippocampal place-cell replay with spatial reward propagation

Author

Listed:
  • Nicolas Cazin
  • Martin Llofriu Alonso
  • Pablo Scleidorovich Chiodi
  • Tatiana Pelc
  • Bruce Harland
  • Alfredo Weitzenfeld
  • Jean-Marc Fellous
  • Peter Ford Dominey

Abstract

As rats learn to search for multiple sources of food or water in a complex environment, they generate increasingly efficient trajectories between reward sites. Such spatial navigation capacity involves the replay of hippocampal place-cells during awake states, generating small sequences of spatially related place-cell activity that we call “snippets”. These snippets occur primarily during sharp-wave-ripples (SWRs). Here we focus on the role of such replay events, as the animal is learning a traveling salesperson task (TSP) across multiple trials. We hypothesize that snippet replay generates synthetic data that can substantially expand and restructure the experience available and make learning more optimal. We developed a model of snippet generation that is modulated by reward, propagated in the forward and reverse directions. This implements a form of spatial credit assignment for reinforcement learning. We use a biologically motivated computational framework known as ‘reservoir computing’ to model prefrontal cortex (PFC) in sequence learning, in which large pools of prewired neural elements process information dynamically through reverberations. This PFC model consolidates snippets into larger spatial sequences that may be later recalled by subsets of the original sequences. Our simulation experiments provide neurophysiological explanations for two pertinent observations related to navigation. Reward modulation allows the system to reject non-optimal segments of experienced trajectories, and reverse replay allows the system to “learn” trajectories that it has not physically experienced, both of which significantly contribute to the TSP behavior.Author summary: As rats search for multiple sources of food in a complex environment, they generate increasingly efficient trajectories between reward sites, across multiple trials. This spatial navigation optimization behavior can be measured in the laboratory using a traveling salesperson task (TSP). This likely involves the coordinated replay of place-cell “snippets” between successive trials. We hypothesize that “snippets” can be used by the prefrontal cortex (PFC) to implement a form of reward-modulated reinforcement learning. Our simulation experiments provide neurophysiological explanations for two pertinent observations related to navigation. Reward modulation allows the system to reject non-optimal segments of experienced trajectories, and reverse replay allows the system to “learn” trajectories that it has not physically experienced, both of which significantly contribute to the TSP behavior.

Suggested Citation

  • Nicolas Cazin & Martin Llofriu Alonso & Pablo Scleidorovich Chiodi & Tatiana Pelc & Bruce Harland & Alfredo Weitzenfeld & Jean-Marc Fellous & Peter Ford Dominey, 2019. "Reservoir computing model of prefrontal cortex creates novel combinations of previous navigation sequences from hippocampal place-cell replay with spatial reward propagation," PLOS Computational Biology, Public Library of Science, vol. 15(7), pages 1-32, July.
  • Handle: RePEc:plo:pcbi00:1006624
    DOI: 10.1371/journal.pcbi.1006624
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006624
    Download Restriction: no

    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1006624&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pcbi.1006624?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Brad E. Pfeiffer & David J. Foster, 2013. "Hippocampal place-cell sequences depict future paths to remembered goals," Nature, Nature, vol. 497(7447), pages 74-79, May.
    2. Priyamvada Rajasethupathy & Sethuraman Sankaran & James H. Marshel & Christina K. Kim & Emily Ferenczi & Soo Yeun Lee & Andre Berndt & Charu Ramakrishnan & Anna Jaffe & Maisie Lo & Conor Liston & Karl, 2015. "Projections from neocortex mediate top-down control of memory retrieval," Nature, Nature, vol. 526(7575), pages 653-659, October.
    3. Mattia Rigotti & Omri Barak & Melissa R. Warden & Xiao-Jing Wang & Nathaniel D. Daw & Earl K. Miller & Stefano Fusi, 2013. "The importance of mixed selectivity in complex cognitive tasks," Nature, Nature, vol. 497(7451), pages 585-590, May.
    4. David J. Foster & Matthew A. Wilson, 2006. "Reverse replay of behavioural sequences in hippocampal place cells during the awake state," Nature, Nature, vol. 440(7084), pages 680-683, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marta Huelin Gorriz & Masahiro Takigawa & Daniel Bendor, 2023. "The role of experience in prioritizing hippocampal replay," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    2. Zhewei Zhang & Yuji K. Takahashi & Marlian Montesinos-Cartegena & Thorsten Kahnt & Angela J. Langdon & Geoffrey Schoenbaum, 2024. "Expectancy-related changes in firing of dopamine neurons depend on hippocampus," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    3. Anli A. Liu & Simon Henin & Saman Abbaspoor & Anatol Bragin & Elizabeth A. Buffalo & Jordan S. Farrell & David J. Foster & Loren M. Frank & Tamara Gedankien & Jean Gotman & Jennifer A. Guidera & Kari , 2022. "A consensus statement on detection of hippocampal sharp wave ripples and differentiation from other fast oscillations," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    4. Lukas Grossberger & Francesco P Battaglia & Martin Vinck, 2018. "Unsupervised clustering of temporal patterns in high-dimensional neuronal ensembles using a novel dissimilarity measure," PLOS Computational Biology, Public Library of Science, vol. 14(7), pages 1-34, July.
    5. Hefei Guan & Steven J. Middleton & Takafumi Inoue & Thomas J. McHugh, 2021. "Lateralization of CA1 assemblies in the absence of CA3 input," Nature Communications, Nature, vol. 12(1), pages 1-10, December.
    6. J Matthew Mahoney & Ali S Titiz & Amanda E Hernan & Rod C Scott, 2016. "Short-Range Temporal Interactions in Sleep; Hippocampal Spike Avalanches Support a Large Milieu of Sequential Activity Including Replay," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-25, February.
    7. Will D Penny & Peter Zeidman & Neil Burgess, 2013. "Forward and Backward Inference in Spatial Cognition," PLOS Computational Biology, Public Library of Science, vol. 9(12), pages 1-22, December.
    8. Guillaume Viejo & Thomas Cortier & Adrien Peyrache, 2018. "Brain-state invariant thalamo-cortical coordination revealed by non-linear encoders," PLOS Computational Biology, Public Library of Science, vol. 14(3), pages 1-25, March.
    9. Siavash Ahmadi & Takuya Sasaki & Marta Sabariego & Christian Leibold & Stefan Leutgeb & Jill K. Leutgeb, 2025. "Distinct roles of dentate gyrus and medial entorhinal cortex inputs for phase precession and temporal correlations in the hippocampal CA3 area," Nature Communications, Nature, vol. 16(1), pages 1-20, December.
    10. Jan Weber & Anne-Kristin Solbakk & Alejandro O. Blenkmann & Anais Llorens & Ingrid Funderud & Sabine Leske & Pål Gunnar Larsson & Jugoslav Ivanovic & Robert T. Knight & Tor Endestad & Randolph F. Helf, 2024. "Ramping dynamics and theta oscillations reflect dissociable signatures during rule-guided human behavior," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    11. Pierre O. Boucher & Tian Wang & Laura Carceroni & Gary Kane & Krishna V. Shenoy & Chandramouli Chandrasekaran, 2023. "Initial conditions combine with sensory evidence to induce decision-related dynamics in premotor cortex," Nature Communications, Nature, vol. 14(1), pages 1-28, December.
    12. Usman Farooq & George Dragoi, 2024. "Experience of Euclidean geometry sculpts the development and dynamics of rodent hippocampal sequential cell assemblies," Nature Communications, Nature, vol. 15(1), pages 1-24, December.
    13. Wenyi Zhang & Yang Xie & Tianming Yang, 2022. "Reward salience but not spatial attention dominates the value representation in the orbitofrontal cortex," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    14. Hagai Lalazar & L F Abbott & Eilon Vaadia, 2016. "Tuning Curves for Arm Posture Control in Motor Cortex Are Consistent with Random Connectivity," PLOS Computational Biology, Public Library of Science, vol. 12(5), pages 1-27, May.
    15. Toon Van de Maele & Bart Dhoedt & Tim Verbelen & Giovanni Pezzulo, 2024. "A hierarchical active inference model of spatial alternation tasks and the hippocampal-prefrontal circuit," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    16. Jason S Prentice & Olivier Marre & Mark L Ioffe & Adrianna R Loback & Gašper Tkačik & Michael J Berry II, 2016. "Error-Robust Modes of the Retinal Population Code," PLOS Computational Biology, Public Library of Science, vol. 12(11), pages 1-32, November.
    17. Chaogan Yan & Dongqiang Liu & Yong He & Qihong Zou & Chaozhe Zhu & Xinian Zuo & Xiangyu Long & Yufeng Zang, 2009. "Spontaneous Brain Activity in the Default Mode Network Is Sensitive to Different Resting-State Conditions with Limited Cognitive Load," PLOS ONE, Public Library of Science, vol. 4(5), pages 1-11, May.
    18. Rishi Rajalingham & Hansem Sohn & Mehrdad Jazayeri, 2025. "Dynamic tracking of objects in the macaque dorsomedial frontal cortex," Nature Communications, Nature, vol. 16(1), pages 1-16, December.
    19. Robert N. Fetcho & Baila S. Hall & David J. Estrin & Alexander P. Walsh & Peter J. Schuette & Jesse Kaminsky & Ashna Singh & Jacob Roshgodal & Charlotte C. Bavley & Viraj Nadkarni & Susan Antigua & Th, 2023. "Regulation of social interaction in mice by a frontostriatal circuit modulated by established hierarchical relationships," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    20. Hong Yu & Xinkuan Xiang & Zongming Chen & Xu Wang & Jiaqi Dai & Xinxin Wang & Pengcheng Huang & Zheng-dong Zhao & Wei L. Shen & Haohong Li, 2021. "Periaqueductal gray neurons encode the sequential motor program in hunting behavior of mice," Nature Communications, Nature, vol. 12(1), pages 1-15, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1006624. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.