IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0226789.html
   My bibliography  Save this article

Quantifying exploration in reward-based motor learning

Author

Listed:
  • Nina M van Mastrigt
  • Jeroen B J Smeets
  • Katinka van der Kooij

Abstract

Exploration in reward-based motor learning is observable in experimental data as increased variability. In order to quantify exploration, we compare three methods for estimating other sources of variability: sensorimotor noise. We use a task in which participants could receive stochastic binary reward feedback following a target-directed weight shift. Participants first performed six baseline blocks without feedback, and next twenty blocks alternating with and without feedback. Variability was assessed based on trial-to-trial changes in movement endpoint. We estimated sensorimotor noise by the median squared trial-to-trial change in movement endpoint for trials in which no exploration is expected. We identified three types of such trials: trials in baseline blocks, trials in the blocks without feedback, and rewarded trials in the blocks with feedback. We estimated exploration by the median squared trial-to-trial change following non-rewarded trials minus sensorimotor noise. As expected, variability was larger following non-rewarded trials than following rewarded trials. This indicates that our reward-based weight-shifting task successfully induced exploration. Most importantly, our three estimates of sensorimotor noise differed: the estimate based on rewarded trials was significantly lower than the estimates based on the two types of trials without feedback. Consequently, the estimates of exploration also differed. We conclude that the quantification of exploration depends critically on the type of trials used to estimate sensorimotor noise. We recommend the use of variability following rewarded trials.

Suggested Citation

  • Nina M van Mastrigt & Jeroen B J Smeets & Katinka van der Kooij, 2020. "Quantifying exploration in reward-based motor learning," PLOS ONE, Public Library of Science, vol. 15(4), pages 1-14, April.
  • Handle: RePEc:plo:pone00:0226789
    DOI: 10.1371/journal.pone.0226789
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0226789
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0226789&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0226789?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kang He & You Liang & Farnaz Abdollahi & Moria Fisher Bittmann & Konrad Kording & Kunlin Wei, 2016. "The Statistical Determinants of the Speed of Motor Learning," PLOS Computational Biology, Public Library of Science, vol. 12(9), pages 1-20, September.
    2. Evren C. Tumer & Michael S. Brainard, 2007. "Performance variability enables adaptive plasticity of ‘crystallized’ adult birdsong," Nature, Nature, vol. 450(7173), pages 1240-1244, December.
    3. Xiuli Chen & Kieran Mohr & Joseph M Galea, 2017. "Predicting explorative motor learning using decision-making and motor noise," PLOS Computational Biology, Public Library of Science, vol. 13(4), pages 1-33, April.
    4. Joshua G A Cashaback & Christopher K Lao & Dimitrios J Palidis & Susan K Coltman & Heather R McGregor & Paul L Gribble, 2019. "The gradient of the reinforcement landscape influences sensorimotor learning," PLOS Computational Biology, Public Library of Science, vol. 15(3), pages 1-27, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Joshua G A Cashaback & Christopher K Lao & Dimitrios J Palidis & Susan K Coltman & Heather R McGregor & Paul L Gribble, 2019. "The gradient of the reinforcement landscape influences sensorimotor learning," PLOS Computational Biology, Public Library of Science, vol. 15(3), pages 1-27, March.
    2. Xiuli Chen & Robb B Rutledge & Harriet R Brown & Raymond J Dolan & Sven Bestmann & Joseph M Galea, 2018. "Age-dependent Pavlovian biases influence motor decision-making," PLOS Computational Biology, Public Library of Science, vol. 14(7), pages 1-22, July.
    3. Nicoletti, Giuseppe & von Rueden, Christina & Andrews, Dan, 2020. "Digital technology diffusion: A matter of capabilities, incentives or both?," European Economic Review, Elsevier, vol. 128(C).
    4. Sravani Kondapavulur & Stefan M. Lemke & David Darevsky & Ling Guo & Preeya Khanna & Karunesh Ganguly, 2022. "Transition from predictable to variable motor cortex and striatal ensemble patterning during behavioral exploration," Nature Communications, Nature, vol. 13(1), pages 1-17, December.
    5. Fabian Heim & Ezequiel Mendoza & Avani Koparkar & Daniela Vallentin, 2024. "Disinhibition enables vocal repertoire expansion after a critical period," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    6. Takuto Kawaji & Mizuki Fujibayashi & Kentaro Abe, 2024. "Goal-directed and flexible modulation of syllable sequence within birdsong," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    7. Taghizadeh-Hesary, Farhad & Yoshino, Naoyuki & Fukuda, Lisa & Rasoulinezhad, Ehsan, 2021. "A model for calculating optimal credit guarantee fee for small and medium-sized enterprises," Economic Modelling, Elsevier, vol. 95(C), pages 361-373.
    8. Jonathan B Dingwell & Joby John & Joseph P Cusumano, 2010. "Do Humans Optimally Exploit Redundancy to Control Step Variability in Walking?," PLOS Computational Biology, Public Library of Science, vol. 6(7), pages 1-15, July.
    9. Ignasi Cos, 2017. "Perceived effort for motor control and decision-making," PLOS Biology, Public Library of Science, vol. 15(8), pages 1-6, August.
    10. Julie E Miller & Austin T Hilliard & Stephanie A White, 2010. "Song Practice Promotes Acute Vocal Variability at a Key Stage of Sensorimotor Learning," PLOS ONE, Public Library of Science, vol. 5(1), pages 1-15, January.
    11. Hanan Shteingart & Yonatan Loewenstein, 2014. "Reinforcement Learning and Human Behavior," Discussion Paper Series dp656, The Federmann Center for the Study of Rationality, the Hebrew University, Jerusalem.
    12. Xiuli Chen & Kieran Mohr & Joseph M Galea, 2017. "Predicting explorative motor learning using decision-making and motor noise," PLOS Computational Biology, Public Library of Science, vol. 13(4), pages 1-33, April.
    13. Simily Sabu & Arianna Curioni & Cordula Vesper & Natalie Sebanz & Günther Knoblich, 2020. "How does a partner’s motor variability affect joint action?," PLOS ONE, Public Library of Science, vol. 15(10), pages 1-24, October.
    14. Daniel Blustein & Ahmed Shehata & Kevin Englehart & Jonathon Sensinger, 2018. "Conventional analysis of trial-by-trial adaptation is biased: Empirical and theoretical support using a Bayesian estimator," PLOS Computational Biology, Public Library of Science, vol. 14(12), pages 1-15, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0226789. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.