IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v14y2023i1d10.1038_s41467-023-44248-1.html
   My bibliography  Save this article

Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex

Author

Listed:
  • Colin W. Hoy

    (University of California, San Francisco
    University of California, Berkeley)

  • David R. Quiroga-Martinez

    (University of California, Berkeley
    Aarhus University & The Royal Academy of Music)

  • Eduardo Sandoval

    (University of California, Berkeley)

  • David King-Stephens

    (California Pacific Medical Center
    Yale School of Medicine)

  • Kenneth D. Laxer

    (California Pacific Medical Center)

  • Peter Weber

    (California Pacific Medical Center)

  • Jack J. Lin

    (University of California, Davis
    University of California, Davis)

  • Robert T. Knight

    (University of California, Berkeley
    University of California, Berkeley)

Abstract

The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.

Suggested Citation

  • Colin W. Hoy & David R. Quiroga-Martinez & Eduardo Sandoval & David King-Stephens & Kenneth D. Laxer & Peter Weber & Jack J. Lin & Robert T. Knight, 2023. "Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
  • Handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-44248-1
    DOI: 10.1038/s41467-023-44248-1
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-023-44248-1
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-023-44248-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ian R. Kleckner & Jiahe Zhang & Alexandra Touroutoglou & Lorena Chanes & Chenjie Xia & W. Kyle Simmons & Karen S. Quigley & Bradford C. Dickerson & Lisa Feldman Barrett, 2017. "Evidence for a large-scale brain system supporting allostasis and interoception in humans," Nature Human Behaviour, Nature, vol. 1(5), pages 1-14, May.
    2. You-Ping Yang & Xinjian Li & Veit Stuphorn, 2022. "Primate anterior insular cortex represents economic decision variables proposed by prospect theory," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    3. Masayuki Matsumoto & Okihide Hikosaka, 2009. "Two types of dopamine neuron distinctly convey positive and negative motivational signals," Nature, Nature, vol. 459(7248), pages 837-841, June.
    4. Jiefeng Jiang & Jeffrey Beck & Katherine Heller & Tobias Egner, 2015. "An insula-frontostriatal network mediates flexible cognitive control by adaptively predicting changing control demands," Nature Communications, Nature, vol. 6(1), pages 1-11, November.
    5. Will Dabney & Zeb Kurth-Nelson & Naoshige Uchida & Clara Kwon Starkweather & Demis Hassabis & Rémi Munos & Matthew Botvinick, 2020. "A distributional code for value in dopamine-based reinforcement learning," Nature, Nature, vol. 577(7792), pages 671-675, January.
    6. Romy Frömer & Carolyn K. Dean Wolf & Amitai Shenhav, 2019. "Goal congruency dominates reward value in accounting for behavioral and neural correlates of value-based decision-making," Nature Communications, Nature, vol. 10(1), pages 1-11, December.
    7. Marco K. Wittmann & Elsa Fouragnan & Davide Folloni & Miriam C. Klein-Flügge & Bolton K. H. Chau & Mehdi Khamassi & Matthew F. S. Rushworth, 2020. "Global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys," Nature Communications, Nature, vol. 11(1), pages 1-17, December.
    8. Ben Engelhard & Joel Finkelstein & Julia Cox & Weston Fleming & Hee Jae Jang & Sharon Ornelas & Sue Ann Koay & Stephan Y. Thiberge & Nathaniel D. Daw & David W. Tank & Ilana B. Witten, 2019. "Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons," Nature, Nature, vol. 570(7762), pages 509-513, June.
    9. Weidong Cai & Srikanth Ryali & Ramkrishna Pasumarthy & Viswanath Talasila & Vinod Menon, 2021. "Dynamic causal brain circuits during working memory and their functional controllability," Nature Communications, Nature, vol. 12(1), pages 1-16, December.
    10. Ilya E. Monosov, 2017. "Anterior cingulate is a source of valence-specific information about value and uncertainty," Nature Communications, Nature, vol. 8(1), pages 1-12, December.
    11. Erin L. Rich & Joni D. Wallis, 2017. "Spatiotemporal dynamics of information encoding revealed in orbitofrontal high-gamma," Nature Communications, Nature, vol. 8(1), pages 1-13, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vincent Man & Jeffrey Cockburn & Oliver Flouty & Phillip E. Gander & Masahiro Sawada & Christopher K. Kovach & Hiroto Kawasaki & Hiroyuki Oya & Matthew A. Howard III & John P. O’Doherty, 2024. "Temporally organized representations of reward and risk in the human brain," Nature Communications, Nature, vol. 15(1), pages 1-16, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wan-Yu Shih & Hsiang-Yu Yu & Cheng-Chia Lee & Chien-Chen Chou & Chien Chen & Paul W. Glimcher & Shih-Wei Wu, 2023. "Electrophysiological population dynamics reveal context dependencies during decision making in human frontal cortex," Nature Communications, Nature, vol. 14(1), pages 1-24, December.
    2. Torben Ott & Anna Marlina Stein & Andreas Nieder, 2023. "Dopamine receptor activation regulates reward expectancy signals during cognitive control in primate prefrontal neurons," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    3. Leo Chi U Seak & Simone Ferrari-Toniolo & Ritesh Jain & Kirby Nielsen & Wolfram Schultz, 2023. "Systematic comparison of risky choices in humans and monkeys," Working Papers 202316, University of Liverpool, Department of Economics.
    4. Min Jung Kim & Daniel J. Gibson & Dan Hu & Tomoko Yoshida & Emily Hueske & Ayano Matsushima & Ara Mahar & Cynthia J. Schofield & Patlapa Sompolpong & Kathy T. Tran & Lin Tian & Ann M. Graybiel, 2024. "Dopamine release plateau and outcome signals in dorsal striatum contrast with classic reinforcement learning formulations," Nature Communications, Nature, vol. 15(1), pages 1-20, December.
    5. Wenqi Chen & Jiejunyi Liang & Qiyun Wu & Yunyun Han, 2024. "Anterior cingulate cortex provides the neural substrates for feedback-driven iteration of decision and value representation," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    6. Patricia L. Lockwood & Jo Cutler & Daniel Drew & Ayat Abdurahman & Deva Sanjeeva Jeyaretna & Matthew A. J. Apps & Masud Husain & Sanjay G. Manohar, 2024. "Human ventromedial prefrontal cortex is necessary for prosocial motivation," Nature Human Behaviour, Nature, vol. 8(7), pages 1403-1416, July.
    7. Jan Weber & Anne-Kristin Solbakk & Alejandro O. Blenkmann & Anais Llorens & Ingrid Funderud & Sabine Leske & Pål Gunnar Larsson & Jugoslav Ivanovic & Robert T. Knight & Tor Endestad & Randolph F. Helf, 2024. "Ramping dynamics and theta oscillations reflect dissociable signatures during rule-guided human behavior," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    8. Kazuo Sano, 2022. "New Concept for the Value Function of Prospect Theory," Papers 2211.00131, arXiv.org, revised Jun 2024.
    9. Frederick Callaway & Antonio Rangel & Thomas L Griffiths, 2021. "Fixation patterns in simple choice reflect optimal information sampling," PLOS Computational Biology, Public Library of Science, vol. 17(3), pages 1-29, March.
    10. Ethan Trepka & Mehran Spitmaan & Bilal A. Bari & Vincent D. Costa & Jeremiah Y. Cohen & Alireza Soltani, 2021. "Entropy-based metrics for predicting choice behavior based on local response to reward," Nature Communications, Nature, vol. 12(1), pages 1-16, December.
    11. Hiroyuki Kawai & Youcef Bouchekioua & Naoya Nishitani & Kazuhei Niitani & Shoma Izumi & Hinako Morishita & Chihiro Andoh & Yuma Nagai & Masashi Koda & Masako Hagiwara & Koji Toda & Hisashi Shirakawa &, 2022. "Median raphe serotonergic neurons projecting to the interpeduncular nucleus control preference and aversion," Nature Communications, Nature, vol. 13(1), pages 1-22, December.
    12. Dimitrije Marković & Andrea M F Reiter & Stefan J Kiebel, 2019. "Predicting change: Approximate inference under explicit representation of temporal structure in changing environments," PLOS Computational Biology, Public Library of Science, vol. 15(1), pages 1-31, January.
    13. Paul Leon Brown & Paul D Shepard, 2013. "Lesions of the Fasciculus Retroflexus Alter Footshock-Induced cFos Expression in the Mesopontine Rostromedial Tegmental Area of Rats," PLOS ONE, Public Library of Science, vol. 8(4), pages 1-9, April.
    14. Zhewei Zhang & Yuji K. Takahashi & Marlian Montesinos-Cartegena & Thorsten Kahnt & Angela J. Langdon & Geoffrey Schoenbaum, 2024. "Expectancy-related changes in firing of dopamine neurons depend on hippocampus," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    15. Nir Moneta & Mona M. Garvert & Hauke R. Heekeren & Nicolas W. Schuck, 2023. "Task state representations in vmPFC mediate relevant and irrelevant value signals and their behavioral influence," Nature Communications, Nature, vol. 14(1), pages 1-21, December.
    16. Huanyuan Zhou & KongFatt Wong-Lin & Da-Hui Wang, 2018. "Parallel Excitatory and Inhibitory Neural Circuit Pathways Underlie Reward-Based Phasic Neural Responses," Complexity, Hindawi, vol. 2018, pages 1-20, April.
    17. Hong Yu & Xinkuan Xiang & Zongming Chen & Xu Wang & Jiaqi Dai & Xinxin Wang & Pengcheng Huang & Zheng-dong Zhao & Wei L. Shen & Haohong Li, 2021. "Periaqueductal gray neurons encode the sequential motor program in hunting behavior of mice," Nature Communications, Nature, vol. 12(1), pages 1-15, December.
    18. John N. J. Reynolds & Riccardo Avvisati & Paul D. Dodson & Simon D. Fisher & Manfred J. Oswald & Jeffery R. Wickens & Yan-Feng Zhang, 2022. "Coincidence of cholinergic pauses, dopaminergic activation and depolarisation of spiny projection neurons drives synaptic plasticity in the striatum," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    19. Yosuke Yawata & Yu Shikano & Jun Ogasawara & Kenichi Makino & Tetsuhiko Kashima & Keiko Ihara & Airi Yoshimoto & Shota Morikawa & Sho Yagishita & Kenji F. Tanaka & Yuji Ikegaya, 2023. "Mesolimbic dopamine release precedes actively sought aversive stimuli in mice," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    20. Ali Ghazizadeh & Okihide Hikosaka, 2022. "Salience memories formed by value, novelty and aversiveness jointly shape object responses in the prefrontal cortex and basal ganglia," Nature Communications, Nature, vol. 13(1), pages 1-14, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-44248-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.