IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v14y2023i1d10.1038_s41467-023-39536-9.html
   My bibliography  Save this article

Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance

Author

Listed:
  • Taisei Sugiyama

    (University of Tsukuba)

  • Nicolas Schweighofer

    (University of Southern California)

  • Jun Izawa

    (University of Tsukuba)

Abstract

Humans and animals develop learning-to-learn strategies throughout their lives to accelerate learning. One theory suggests that this is achieved by a metacognitive process of controlling and monitoring learning. Although such learning-to-learn is also observed in motor learning, the metacognitive aspect of learning regulation has not been considered in classical theories of motor learning. Here, we formulated a minimal mechanism of this process as reinforcement learning of motor learning properties, which regulates a policy for memory update in response to sensory prediction error while monitoring its performance. This theory was confirmed in human motor learning experiments, in which the subjective sense of learning-outcome association determined the direction of up- and down-regulation of both learning speed and memory retention. Thus, it provides a simple, unifying account for variations in learning speeds, where the reinforcement learning mechanism monitors and controls the motor learning process.

Suggested Citation

  • Taisei Sugiyama & Nicolas Schweighofer & Jun Izawa, 2023. "Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
  • Handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-39536-9
    DOI: 10.1038/s41467-023-39536-9
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-023-39536-9
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-023-39536-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bates, Douglas & Mächler, Martin & Bolker, Ben & Walker, Steve, 2015. "Fitting Linear Mixed-Effects Models Using lme4," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i01).
    2. Tomaso Poggio & Emilio Bizzi, 2004. "Generalization in vision and motor control," Nature, Nature, vol. 431(7010), pages 768-774, October.
    3. Andrew Gelman & Christian Hennig, 2017. "Beyond subjective and objective in statistics," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(4), pages 967-1033, October.
    4. James B. Heald & Máté Lengyel & Daniel M. Wolpert, 2021. "Contextual inference underlies the learning of sensorimotor repertoires," Nature, Nature, vol. 600(7889), pages 489-493, December.
    5. Martha L. Streng & Laurentiu S. Popa & Timothy J. Ebner, 2018. "Modulation of sensory prediction error in Purkinje cells during visual feedback manipulations," Nature Communications, Nature, vol. 9(1), pages 1-12, December.
    6. Mark J. Wagner & Tony Hyun Kim & Joan Savall & Mark J. Schnitzer & Liqun Luo, 2017. "Cerebellar granule cells encode the expectation of reward," Nature, Nature, vol. 544(7648), pages 96-100, April.
    7. Kurt A. Thoroughman & Reza Shadmehr, 2000. "Learning of action through adaptive combination of motor primitives," Nature, Nature, vol. 407(6805), pages 742-747, October.
    8. Sungshin Kim & Kenji Ogawa & Jinchi Lv & Nicolas Schweighofer & Hiroshi Imamizu, 2015. "Neural Substrates Related to Motor Memory with Multiple Timescales in Sensorimotor Adaptation," PLOS Biology, Public Library of Science, vol. 13(12), pages 1-23, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yael Mandelblat-Cerf & Itai Novick & Eilon Vaadia, 2011. "Expressions of Multiple Neuronal Dynamics during Sensorimotor Learning in the Motor Cortex of Behaving Monkeys," PLOS ONE, Public Library of Science, vol. 6(7), pages 1-14, July.
    2. Ellen Boven & Joseph Pemberton & Paul Chadderton & Richard Apps & Rui Ponte Costa, 2023. "Cerebro-cerebellar networks facilitate learning through feedback decoupling," Nature Communications, Nature, vol. 14(1), pages 1-18, December.
    3. JANSSENS, Jochen & DE CORTE, Annelies & SÖRENSEN, Kenneth, 2016. "Water distribution network design optimisation with respect to reliability," Working Papers 2016007, University of Antwerp, Faculty of Business and Economics.
    4. Mayo, Deborah, 2024. "Error statistics, Bayes-factor Tests and the Fallacy of Non-exhaustive Alternatives," OSF Preprints tmgqd, Center for Open Science.
    5. Raymond Hernandez & Elizabeth A. Pyatak & Cheryl L. P. Vigen & Haomiao Jin & Stefan Schneider & Donna Spruijt-Metz & Shawn C. Roll, 2021. "Understanding Worker Well-Being Relative to High-Workload and Recovery Activities across a Whole Day: Pilot Testing an Ecological Momentary Assessment Technique," IJERPH, MDPI, vol. 18(19), pages 1-17, October.
    6. Elisabeth Beckmann & Lukas Olbrich & Joseph Sakshaug, 2024. "Multivariate assessment of interviewer-related errors in a cross-national economic survey (Lukas Olbrich, Elisabeth Beckmann, Joseph W. Sakshaug)," Working Papers 253, Oesterreichische Nationalbank (Austrian Central Bank).
    7. Valentina Krenz & Arjen Alink & Tobias Sommer & Benno Roozendaal & Lars Schwabe, 2023. "Time-dependent memory transformation in hippocampus and neocortex is semantic in nature," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    8. Morán-Ordóñez, Alejandra & Ameztegui, Aitor & De Cáceres, Miquel & de-Miguel, Sergio & Lefèvre, François & Brotons, Lluís & Coll, Lluís, 2020. "Future trade-offs and synergies among ecosystem services in Mediterranean forests under global change scenarios," Ecosystem Services, Elsevier, vol. 45(C).
    9. Damian M. Herz & Manuel Bange & Gabriel Gonzalez-Escamilla & Miriam Auer & Keyoumars Ashkan & Petra Fischer & Huiling Tan & Rafal Bogacz & Muthuraman Muthuraman & Sergiu Groppa & Peter Brown, 2022. "Dynamic control of decision and movement speed in the human basal ganglia," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    10. Dongyan Liu & Chongran Zhou & John K. Keesing & Oscar Serrano & Axel Werner & Yin Fang & Yingjun Chen & Pere Masque & Janine Kinloch & Aleksey Sadekov & Yan Du, 2022. "Wildfires enhance phytoplankton production in tropical oceans," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    11. Zhaogeng Yang & Yanhui Li & Peijin Hu & Jun Ma & Yi Song, 2020. "Prevalence of Anemia and its Associated Factors among Chinese 9-, 12-, and 14-Year-Old Children: Results from 2014 Chinese National Survey on Students Constitution and Health," IJERPH, MDPI, vol. 17(5), pages 1-10, February.
    12. Marco Lopez-Cruz & Fernando M. Aguate & Jacob D. Washburn & Natalia Leon & Shawn M. Kaeppler & Dayane Cristina Lima & Ruijuan Tan & Addie Thompson & Laurence Willard Bretonne & Gustavo los Campos, 2023. "Leveraging data from the Genomes-to-Fields Initiative to investigate genotype-by-environment interactions in maize in North America," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    13. Baumann, Elias & Kern, Jana & Lessmann, Stefan, 2019. "Usage Continuance in Software-as-a-Service," IRTG 1792 Discussion Papers 2019-005, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".
    14. repec:cup:judgdm:v:16:y:2021:i:1:p:201-237 is not listed on IDEAS
    15. C. Gabriel Hidalgo Pizango & Eurídice N. Honorio Coronado & Jhon del Águila-Pasquel & Gerardo Flores Llampazo & Johan de Jong & César J. Córdova Oroche & José M. Reyna Huaymacari & Steve J. Carver & D, 2022. "Sustainable palm fruit harvesting as a pathway to conserve Amazon peatland forests," Nature Sustainability, Nature, vol. 5(6), pages 479-487, June.
    16. Evans O. Mudibo & Jasper Bogaert & Caroline Tigoi & Moses M. Ngari & Benson O. Singa & Christina L. Lancioni & Abdoulaye Hama Diallo & Emmie Mbale & Ezekiel Mupere & John Mukisa & Johnstone Thitiri & , 2024. "Systemic biological mechanisms underpin poor post-discharge growth among severely wasted children with HIV," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    17. Lin-Lin Wang & Zachary Y. Huang & Wen-Fei Dai & Yong-Ping Yang & Yuan-Wen Duan, 2024. "Mixed effects of honey bees on pollination function in the Tibetan alpine grasslands," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    18. Szefer Elena & Lu Donghuan & Nathoo Farouk & Beg Mirza Faisal & Graham Jinko, 2017. "Multivariate association between single-nucleotide polymorphisms in Alzgene linkage regions and structural changes in the brain: discovery, refinement and validation," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 16(5-6), pages 367-386, December.
    19. Julien Collet & Samantha C Patrick & Henri Weimerskirch, 2017. "A comparative analysis of the behavioral response to fishing boats in two albatross species," Behavioral Ecology, International Society for Behavioral Ecology, vol. 28(5), pages 1337-1347.
    20. Sean Coogan & Zhixian Sui & David Raubenheimer, 2018. "Gluttony and guilt: monthly trends in internet search query data are comparable with national-level energy intake and dieting behavior," Palgrave Communications, Palgrave Macmillan, vol. 4(1), pages 1-9, December.
    21. Darcy Steeg Morris & Kimberly F. Sellers, 2022. "A Flexible Mixed Model for Clustered Count Data," Stats, MDPI, vol. 5(1), pages 1-18, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-39536-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.