IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-53176-7.html
   My bibliography  Save this article

Dopamine release plateau and outcome signals in dorsal striatum contrast with classic reinforcement learning formulations

Author

Listed:
  • Min Jung Kim

    (Massachusetts Institute of Technology
    Southwestern Medical Center)

  • Daniel J. Gibson

    (Massachusetts Institute of Technology)

  • Dan Hu

    (Massachusetts Institute of Technology)

  • Tomoko Yoshida

    (Massachusetts Institute of Technology)

  • Emily Hueske

    (Massachusetts Institute of Technology)

  • Ayano Matsushima

    (Massachusetts Institute of Technology)

  • Ara Mahar

    (Massachusetts Institute of Technology)

  • Cynthia J. Schofield

    (Massachusetts Institute of Technology
    Johns Hopkins University School of Medicine)

  • Patlapa Sompolpong

    (Massachusetts Institute of Technology
    Icahn School of Medicine at Mount Sinai)

  • Kathy T. Tran

    (Massachusetts Institute of Technology)

  • Lin Tian

    (Max Planck Florida Institute for Neuroscience)

  • Ann M. Graybiel

    (Massachusetts Institute of Technology)

Abstract

We recorded dopamine release signals in centromedial and centrolateral sectors of the striatum as mice learned consecutive versions of visual cue-outcome conditioning tasks. Dopamine release responses differed for the centromedial and centrolateral sites. In neither sector could these be accounted for by classic reinforcement learning alone as classically applied to the activity of nigral dopamine-containing neurons. Medially, cue responses ranged from initial sharp peaks to modulated plateau responses; outcome (reward) responses during cue conditioning were minimal or, initially, negative. At centrolateral sites, by contrast, strong, transient dopamine release responses occurred at both cue and outcome. Prolonged, plateau release responses to cues emerged in both regions when discriminative behavioral responses became required. At most sites, we found no evidence for a transition from outcome signaling to cue signaling, a hallmark of temporal difference reinforcement learning as applied to midbrain dopaminergic neuronal activity. These findings delineate a reshaping of striatal dopamine release activity during learning and suggest that current views of reward prediction error encoding need review to accommodate distinct learning-related spatial and temporal patterns of striatal dopamine release in the dorsal striatum.

Suggested Citation

  • Min Jung Kim & Daniel J. Gibson & Dan Hu & Tomoko Yoshida & Emily Hueske & Ayano Matsushima & Ara Mahar & Cynthia J. Schofield & Patlapa Sompolpong & Kathy T. Tran & Lin Tian & Ann M. Graybiel, 2024. "Dopamine release plateau and outcome signals in dorsal striatum contrast with classic reinforcement learning formulations," Nature Communications, Nature, vol. 15(1), pages 1-20, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-53176-7
    DOI: 10.1038/s41467-024-53176-7
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-53176-7
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-53176-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ali Mohebi & Jeffrey R. Pettibone & Arif A. Hamid & Jenny-Marie T. Wong & Leah T. Vinson & Tommaso Patriarchi & Lin Tian & Robert T. Kennedy & Joshua D. Berke, 2019. "Dissociable dopamine dynamics for learning and motivation," Nature, Nature, vol. 570(7759), pages 65-70, June.
    2. Neir Eshel & Michael Bukwich & Vinod Rao & Vivian Hemmelder & Ju Tian & Naoshige Uchida, 2015. "Arithmetic and local circuitry underlying dopamine prediction errors," Nature, Nature, vol. 525(7568), pages 243-246, September.
    3. Masayuki Matsumoto & Okihide Hikosaka, 2009. "Two types of dopamine neuron distinctly convey positive and negative motivational signals," Nature, Nature, vol. 459(7248), pages 837-841, June.
    4. Anne C. Krok & Marta Maltese & Pratik Mistry & Xiaolei Miao & Yulong Li & Nicolas X. Tritsch, 2023. "Intrinsic dopamine and acetylcholine dynamics in the striatum of mice," Nature, Nature, vol. 621(7979), pages 543-549, September.
    5. Paul E. M. Phillips & Garret D. Stuber & Michael L. A. V. Heien & R. Mark Wightman & Regina M. Carelli, 2003. "Erratum: Subsecond dopamine release promotes cocaine seeking," Nature, Nature, vol. 423(6938), pages 461-461, May.
    6. Jeffrey E. Markowitz & Winthrop F. Gillis & Maya Jay & Jeffrey Wood & Ryley W. Harris & Robert Cieszkowski & Rebecca Scott & David Brann & Dorothy Koveal & Tomasz Kula & Caleb Weinreb & Mohammed Abdal, 2023. "Spontaneous behaviour is structured by reinforcement without explicit reward," Nature, Nature, vol. 614(7946), pages 108-117, February.
    7. Lynne Chantranupong & Celia C. Beron & Joshua A. Zimmer & Michelle J. Wen & Wengang Wang & Bernardo L. Sabatini, 2023. "Dopamine and glutamate regulate striatal acetylcholine in decision-making," Nature, Nature, vol. 621(7979), pages 577-585, September.
    8. Ben Engelhard & Joel Finkelstein & Julia Cox & Weston Fleming & Hee Jae Jang & Sharon Ornelas & Sue Ann Koay & Stephan Y. Thiberge & Nathaniel D. Daw & David W. Tank & Ilana B. Witten, 2019. "Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons," Nature, Nature, vol. 570(7762), pages 509-513, June.
    9. Ali Mohebi & Jeffrey R. Pettibone & Arif A. Hamid & Jenny-Marie T. Wong & Leah T. Vinson & Tommaso Patriarchi & Lin Tian & Robert T. Kennedy & Joshua D. Berke, 2019. "Publisher Correction: Dissociable dopamine dynamics for learning and motivation," Nature, Nature, vol. 571(7763), pages 3-3, July.
    10. Paul E. M. Phillips & Garret D. Stuber & Michael L. A. V. Heien & R. Mark Wightman & Regina M. Carelli, 2003. "Subsecond dopamine release promotes cocaine seeking," Nature, Nature, vol. 422(6932), pages 614-618, April.
    11. M. W. Howe & D. A. Dombeck, 2016. "Rapid signalling in distinct dopaminergic axons during locomotion and reward," Nature, Nature, vol. 535(7613), pages 505-510, July.
    12. Mark W. Howe & Patrick L. Tierney & Stefan G. Sandberg & Paul E. M. Phillips & Ann M. Graybiel, 2013. "Prolonged dopamine signalling in striatum signals proximity and value of distant rewards," Nature, Nature, vol. 500(7464), pages 575-579, August.
    13. Luke T. Coddington & Sarah E. Lindo & Joshua T. Dudman, 2023. "Mesolimbic dopamine adapts the rate of learning from action," Nature, Nature, vol. 614(7947), pages 294-302, February.
    14. Neir Eshel & Michael Bukwich & Vinod Rao & Vivian Hemmelder & Ju Tian & Naoshige Uchida, 2015. "Erratum: Arithmetic and local circuitry underlying dopamine prediction errors," Nature, Nature, vol. 527(7578), pages 398-398, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ayaka Kato & Kenji Morita, 2016. "Forgetting in Reinforcement Learning Links Sustained Dopamine Signals to Motivation," PLOS Computational Biology, Public Library of Science, vol. 12(10), pages 1-41, October.
    2. Lior Matityahu & Naomi Gilin & Gideon A. Sarpong & Yara Atamna & Lior Tiroshi & Nicolas X. Tritsch & Jeffery R. Wickens & Joshua A. Goldberg, 2023. "Acetylcholine waves and dopamine release in the striatum," Nature Communications, Nature, vol. 14(1), pages 1-23, December.
    3. Lauren Faget & Lucie Oriol & Wen-Chun Lee & Vivien Zell & Cody Sargent & Andrew Flores & Nick G. Hollon & Dhakshin Ramanathan & Thomas S. Hnasko, 2024. "Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types," Nature Communications, Nature, vol. 15(1), pages 1-18, December.
    4. Armando G. Salinas & Jeong Oen Lee & Shana M. Augustin & Shiliang Zhang & Tommaso Patriarchi & Lin Tian & Marisela Morales & Yolanda Mateo & David M. Lovinger, 2023. "Distinct sub-second dopamine signaling in dorsolateral striatum measured by a genetically-encoded fluorescent sensor," Nature Communications, Nature, vol. 14(1), pages 1-16, December.
    5. Abigail Kalmbach & Vanessa Winiger & Nuri Jeong & Arun Asok & Charles R. Gallistel & Peter D. Balsam & Eleanor H. Simpson, 2022. "Dopamine encodes real-time reward availability and transitions between reward availability states on different timescales," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    6. Allen P. F. Chen & Jeffrey M. Malgady & Lu Chen & Kaiyo W. Shi & Eileen Cheng & Joshua L. Plotkin & Shaoyu Ge & Qiaojie Xiong, 2022. "Nigrostriatal dopamine pathway regulates auditory discrimination behavior," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    7. Seetha Krishnan & Chad Heer & Chery Cherian & Mark E. J. Sheffield, 2022. "Reward expectation extinction restructures and degrades CA1 spatial maps through loss of a dopaminergic reward proximity signal," Nature Communications, Nature, vol. 13(1), pages 1-19, December.
    8. Laurens Winkelmeier & Carla Filosa & Renée Hartig & Max Scheller & Markus Sack & Jonathan R. Reinwald & Robert Becker & David Wolf & Martin Fungisai Gerchen & Alexander Sartorius & Andreas Meyer-Linde, 2022. "Striatal hub of dynamic and stabilized prediction coding in forebrain networks for olfactory reinforcement learning," Nature Communications, Nature, vol. 13(1), pages 1-21, December.
    9. Ali Ghazizadeh & Okihide Hikosaka, 2022. "Salience memories formed by value, novelty and aversiveness jointly shape object responses in the prefrontal cortex and basal ganglia," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    10. Allen P. F. Chen & Lu Chen & Kaiyo W. Shi & Eileen Cheng & Shaoyu Ge & Qiaojie Xiong, 2023. "Nigrostriatal dopamine modulates the striatal-amygdala pathway in auditory fear conditioning," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    11. Miguel Á. Luján & Dan P. Covey & Reana Young-Morrison & LanYuan Zhang & Andrew Kim & Fiorella Morgado & Sachin Patel & Caroline E. Bass & Carlos Paladini & Joseph F. Cheer, 2023. "Mobilization of endocannabinoids by midbrain dopamine neurons is required for the encoding of reward prediction," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    12. Xin-Yue Wang & Wen-Bin Jia & Xiang Xu & Rui Chen & Liang-Biao Wang & Xiao-Jing Su & Peng-Fei Xu & Xiao-Qing Liu & Jie Wen & Xiao-Yuan Song & Yuan-Yuan Liu & Zhi Zhang & Xin-Feng Liu & Yan Zhang, 2023. "A glutamatergic DRN–VTA pathway modulates neuropathic pain and comorbid anhedonia-like behavior in mice," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    13. Miguel Skirzewski & Oren Princz-Lebel & Liliana German-Castelan & Alycia M. Crooks & Gerard Kyungwook Kim & Sophie Henke Tarnow & Amy Reichelt & Sara Memar & Daniel Palmer & Yulong Li & R. Jane Rylett, 2022. "Continuous cholinergic-dopaminergic updating in the nucleus accumbens underlies approaches to reward-predicting cues," Nature Communications, Nature, vol. 13(1), pages 1-21, December.
    14. Hachi E. Manzur & Ksenia Vlasov & You-Jhe Jhong & Hung-Yen Chen & Shih-Chieh Lin, 2023. "The behavioral signature of stepwise learning strategy in male rats and its neural correlate in the basal forebrain," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    15. Marie A. Labouesse & Maria Wilhelm & Zacharoula Kagiampaki & Andrew G. Yee & Raphaelle Denis & Masaya Harada & Andrea Gresch & Alina-Măriuca Marinescu & Kanako Otomo & Sebastiano Curreli & Laia Serrat, 2024. "A chemogenetic approach for dopamine imaging with tunable sensitivity," Nature Communications, Nature, vol. 15(1), pages 1-22, December.
    16. Han Guo & Jian-Bo Jiang & Wei Xu & Mu-Tian Zhang & Hui Chen & Huan-Ying Shi & Lu Wang & Miao He & Michael Lazarus & Shan-Qun Li & Zhi-Li Huang & Wei-Min Qu, 2023. "Parasubthalamic calretinin neurons modulate wakefulness associated with exploration in male mice," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    17. John N. J. Reynolds & Riccardo Avvisati & Paul D. Dodson & Simon D. Fisher & Manfred J. Oswald & Jeffery R. Wickens & Yan-Feng Zhang, 2022. "Coincidence of cholinergic pauses, dopaminergic activation and depolarisation of spiny projection neurons drives synaptic plasticity in the striatum," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    18. Yosuke Yawata & Yu Shikano & Jun Ogasawara & Kenichi Makino & Tetsuhiko Kashima & Keiko Ihara & Airi Yoshimoto & Shota Morikawa & Sho Yagishita & Kenji F. Tanaka & Yuji Ikegaya, 2023. "Mesolimbic dopamine release precedes actively sought aversive stimuli in mice," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    19. Zengpeng Han & Nengsong Luo & Wenyu Ma & Xiaodong Liu & Yuxiang Cai & Jiaxin Kou & Jie Wang & Lei Li & Siqi Peng & Zihong Xu & Wen Zhang & Yuxiang Qiu & Yang Wu & Chaohui Ye & Kunzhang Lin & Fuqiang X, 2023. "AAV11 enables efficient retrograde targeting of projection neurons and enhances astrocyte-directed transduction," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    20. Xiaolong Gao & Huan Wei & Wenjie Ma & Wenjie Wu & Wenliang Ji & Junjie Mao & Ping Yu & Lanqun Mao, 2024. "Inflammation-free electrochemical in vivo sensing of dopamine with atomic-level engineered antioxidative single-atom catalyst," Nature Communications, Nature, vol. 15(1), pages 1-12, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-53176-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.