IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1002012.html
   My bibliography  Save this article

Learning from Sensory and Reward Prediction Errors during Motor Adaptation

Author

Listed:
  • Jun Izawa
  • Reza Shadmehr

Abstract

Voluntary motor commands produce two kinds of consequences. Initially, a sensory consequence is observed in terms of activity in our primary sensory organs (e.g., vision, proprioception). Subsequently, the brain evaluates the sensory feedback and produces a subjective measure of utility or usefulness of the motor commands (e.g., reward). As a result, comparisons between predicted and observed consequences of motor commands produce two forms of prediction error. How do these errors contribute to changes in motor commands? Here, we considered a reach adaptation protocol and found that when high quality sensory feedback was available, adaptation of motor commands was driven almost exclusively by sensory prediction errors. This form of learning had a distinct signature: as motor commands adapted, the subjects altered their predictions regarding sensory consequences of motor commands, and generalized this learning broadly to neighboring motor commands. In contrast, as the quality of the sensory feedback degraded, adaptation of motor commands became more dependent on reward prediction errors. Reward prediction errors produced comparable changes in the motor commands, but produced no change in the predicted sensory consequences of motor commands, and generalized only locally. Because we found that there was a within subject correlation between generalization patterns and sensory remapping, it is plausible that during adaptation an individual's relative reliance on sensory vs. reward prediction errors could be inferred. We suggest that while motor commands change because of sensory and reward prediction errors, only sensory prediction errors produce a change in the neural system that predicts sensory consequences of motor commands. Author Summary: It is thought that motor adaptation relies on sensory prediction errors to form an estimate of the perturbation. Here, we present evidence that motor adaptation can be driven by both sensory and reward prediction errors. We found that learning from sensory prediction error altered the predicted consequences of motor commands, leaving behind a sensory remapping, whereas learning from reward prediction error produced comparable change in motor commands, but did not produce a sensory remapping. It is possible that the neural basis of learning from sensory and reward prediction errors are distinct because they produce different generalization patterns.

Suggested Citation

  • Jun Izawa & Reza Shadmehr, 2011. "Learning from Sensory and Reward Prediction Errors during Motor Adaptation," PLOS Computational Biology, Public Library of Science, vol. 7(3), pages 1-11, March.
  • Handle: RePEc:plo:pcbi00:1002012
    DOI: 10.1371/journal.pcbi.1002012
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002012
    Download Restriction: no

    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1002012&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pcbi.1002012?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Terra D. Barnes & Yasuo Kubota & Dan Hu & Dezhe Z. Jin & Ann M. Graybiel, 2005. "Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories," Nature, Nature, vol. 437(7062), pages 1158-1161, October.
    2. Kurt A. Thoroughman & Reza Shadmehr, 2000. "Learning of action through adaptive combination of motor primitives," Nature, Nature, vol. 407(6805), pages 742-747, October.
    3. Maurice A Smith & Ali Ghazizadeh & Reza Shadmehr, 2006. "Interacting Adaptive Processes with Different Timescales Underlie Short-Term Motor Learning," PLOS Biology, Public Library of Science, vol. 4(6), pages 1-1, May.
    4. Christopher M. Harris & Daniel M. Wolpert, 1998. "Signal-dependent noise determines motor planning," Nature, Nature, vol. 394(6695), pages 780-784, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Takuya Honda & Masaya Hirashima & Daichi Nozaki, 2012. "Adaptation to Visual Feedback Delay Influences Visuomotor Learning," PLOS ONE, Public Library of Science, vol. 7(5), pages 1-9, May.
    2. Harry Manley & Peter Dayan & Jörn Diedrichsen, 2014. "When Money Is Not Enough: Awareness, Success, and Variability in Motor Learning," PLOS ONE, Public Library of Science, vol. 9(1), pages 1-11, January.
    3. Ran Darshan & Arthur Leblois & David Hansel, 2014. "Interference and Shaping in Sensorimotor Adaptations with Rewards," PLOS Computational Biology, Public Library of Science, vol. 10(1), pages 1-20, January.
    4. Barbara Feulner & Matthew G. Perich & Lee E. Miller & Claudia Clopath & Juan A. Gallego, 2025. "A neural implementation model of feedback-based motor learning," Nature Communications, Nature, vol. 16(1), pages 1-14, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Takuya Honda & Masaya Hirashima & Daichi Nozaki, 2012. "Adaptation to Visual Feedback Delay Influences Visuomotor Learning," PLOS ONE, Public Library of Science, vol. 7(5), pages 1-9, May.
    2. Joshua G A Cashaback & Heather R McGregor & Ayman Mohatarem & Paul L Gribble, 2017. "Dissociating error-based and reinforcement-based loss functions during sensorimotor learning," PLOS Computational Biology, Public Library of Science, vol. 13(7), pages 1-28, July.
    3. Harry Manley & Peter Dayan & Jörn Diedrichsen, 2014. "When Money Is Not Enough: Awareness, Success, and Variability in Motor Learning," PLOS ONE, Public Library of Science, vol. 9(1), pages 1-11, January.
    4. Barbara Feulner & Matthew G. Perich & Lee E. Miller & Claudia Clopath & Juan A. Gallego, 2025. "A neural implementation model of feedback-based motor learning," Nature Communications, Nature, vol. 16(1), pages 1-14, December.
    5. Abdelhamid Kadiallah & David W Franklin & Etienne Burdet, 2012. "Generalization in Adaptation to Stable and Unstable Dynamics," PLOS ONE, Public Library of Science, vol. 7(10), pages 1-11, October.
    6. Shogo Yonekura & Yasuo Kuniyoshi, 2017. "Bodily motion fluctuation improves reaching success rate in a neurophysical agent via geometric-stochastic resonance," PLOS ONE, Public Library of Science, vol. 12(12), pages 1-16, December.
    7. Shih-Wei Wu & Maria F Dal Martello & Laurence T Maloney, 2009. "Sub-Optimal Allocation of Time in Sequential Movements," PLOS ONE, Public Library of Science, vol. 4(12), pages 1-13, December.
    8. Max Berniker & Megan K O’Brien & Konrad P Kording & Alaa A Ahmed, 2013. "An Examination of the Generalizability of Motor Costs," PLOS ONE, Public Library of Science, vol. 8(1), pages 1-11, January.
    9. Shunta Togo & Takahiro Kagawa & Yoji Uno, 2015. "Control Model for Dampening Hand Vibrations Using Information of Internal and External Coordinates," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-16, April.
    10. Ian S Howard & David W Franklin, 2015. "Neural Tuning Functions Underlie Both Generalization and Interference," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-21, June.
    11. Hugo L Fernandes & Ian H Stevenson & Konrad P Kording, 2012. "Generalization of Stochastic Visuomotor Rotations," PLOS ONE, Public Library of Science, vol. 7(8), pages 1-9, August.
    12. Lionel Rigoux & Emmanuel Guigon, 2012. "A Model of Reward- and Effort-Based Optimal Decision Making and Motor Control," PLOS Computational Biology, Public Library of Science, vol. 8(10), pages 1-13, October.
    13. Yanhao Ren & Qiang Luo & Wenlian Lu, 2023. "Synchronization Analysis of Linearly Coupled Systems with Signal-Dependent Noises," Mathematics, MDPI, vol. 11(10), pages 1-15, May.
    14. Christopher J Hasson & Zhaoran Zhang & Masaki O Abe & Dagmar Sternad, 2016. "Neuromotor Noise Is Malleable by Amplifying Perceived Errors," PLOS Computational Biology, Public Library of Science, vol. 12(8), pages 1-28, August.
    15. Seth W. Egger & Stephen G. Lisberger, 2022. "Neural structure of a sensory decoder for motor control," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    16. Ashesh Vasalya & Gowrishankar Ganesh & Abderrahmane Kheddar, 2018. "More than just co-workers: Presence of humanoid robot co-worker influences human performance," PLOS ONE, Public Library of Science, vol. 13(11), pages 1-19, November.
    17. Cornelis van de Kamp & Peter J Gawthrop & Henrik Gollee & Ian D Loram, 2013. "Refractoriness in Sustained Visuo-Manual Control: Is the Refractory Duration Intrinsic or Does It Depend on External System Properties?," PLOS Computational Biology, Public Library of Science, vol. 9(1), pages 1-15, January.
    18. Josh Merel & Donald M Pianto & John P Cunningham & Liam Paninski, 2015. "Encoder-Decoder Optimization for Brain-Computer Interfaces," PLOS Computational Biology, Public Library of Science, vol. 11(6), pages 1-25, June.
    19. Nidhi Seethapathi & Barrett C. Clark & Manoj Srinivasan, 2024. "Exploration-based learning of a stabilizing controller predicts locomotor adaptation," Nature Communications, Nature, vol. 15(1), pages 1-23, December.
    20. Frank Freyer & James A Roberts & Petra Ritter & Michael Breakspear, 2012. "A Canonical Model of Multistability and Scale-Invariance in Biological Systems," PLOS Computational Biology, Public Library of Science, vol. 8(8), pages 1-15, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1002012. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.