IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0242201.html
   My bibliography  Save this article

Simulating bout-and-pause patterns with reinforcement learning

Author

Listed:
  • Kota Yamada
  • Atsunori Kanemura

Abstract

Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice—an agent makes a choice between operant and other behaviors. The second mechanism is cost—a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.

Suggested Citation

  • Kota Yamada & Atsunori Kanemura, 2020. "Simulating bout-and-pause patterns with reinforcement learning," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-21, November.
  • Handle: RePEc:plo:pone00:0242201
    DOI: 10.1371/journal.pone.0242201
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0242201
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0242201&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0242201?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Carpenter, Bob & Gelman, Andrew & Hoffman, Matthew D. & Lee, Daniel & Goodrich, Ben & Betancourt, Michael & Brubaker, Marcus & Guo, Jiqiang & Li, Peter & Riddell, Allen, 2017. "Stan: A Probabilistic Programming Language," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(i01).
    2. Albert-László Barabási, 2005. "The origin of bursts and heavy tails in human dynamics," Nature, Nature, vol. 435(7039), pages 207-211, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Francis,David C. & Kubinec ,Robert, 2022. "Beyond Political Connections : A Measurement Model Approach to Estimating Firm-levelPolitical Influence in 41 Economies," Policy Research Working Paper Series 10119, The World Bank.
    2. Yongping Bao & Ludwig Danwitz & Fabian Dvorak & Sebastian Fehrler & Lars Hornuf & Hsuan Yu Lin & Bettina von Helversen, 2022. "Similarity and Consistency in Algorithm-Guided Exploration," CESifo Working Paper Series 10188, CESifo.
    3. Torsten Heinrich & Jangho Yang & Shuanping Dai, 2020. "Growth, development, and structural change at the firm-level: The example of the PR China," Papers 2012.14503, arXiv.org.
    4. Xin Xu & Yang Lu & Yupeng Zhou & Zhiguo Fu & Yanjie Fu & Minghao Yin, 2021. "An Information-Explainable Random Walk Based Unsupervised Network Representation Learning Framework on Node Classification Tasks," Mathematics, MDPI, vol. 9(15), pages 1-14, July.
    5. Lu, Xi & Mo, Hongming & Deng, Yong, 2015. "An evidential opinion dynamics model based on heterogeneous social influential power," Chaos, Solitons & Fractals, Elsevier, vol. 73(C), pages 98-107.
    6. Xiaoyue Xi & Simon E. F. Spencer & Matthew Hall & M. Kate Grabowski & Joseph Kagaayi & Oliver Ratmann & Rakai Health Sciences Program and PANGEA‐HIV, 2022. "Inferring the sources of HIV infection in Africa from deep‐sequence data with semi‐parametric Bayesian Poisson flow models," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(3), pages 517-540, June.
    7. Luo, Nanyu & Ji, Feng & Han, Yuting & He, Jinbo & Zhang, Xiaoya, 2024. "Fitting item response theory models using deep learning computational frameworks," OSF Preprints tjxab, Center for Open Science.
    8. Simon DeDeo, 2016. "Conflict and Computation on Wikipedia: A Finite-State Machine Analysis of Editor Interactions," Future Internet, MDPI, vol. 8(3), pages 1-23, July.
    9. Joseph B. Bak-Coleman & Ian Kennedy & Morgan Wack & Andrew Beers & Joseph S. Schafer & Emma S. Spiro & Kate Starbird & Jevin D. West, 2022. "Combining interventions to reduce the spread of viral misinformation," Nature Human Behaviour, Nature, vol. 6(10), pages 1372-1380, October.
    10. David M. Phillippo & Sofia Dias & A. E. Ades & Mark Belger & Alan Brnabic & Alexander Schacht & Daniel Saure & Zbigniew Kadziola & Nicky J. Welton, 2020. "Multilevel network meta‐regression for population‐adjusted treatment comparisons," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(3), pages 1189-1210, June.
    11. Alina Ferecatu & Arnaud Bruyn & Prithwiraj Mukherjee, 2024. "Silently killing your panelists one email at a time: The true cost of email solicitations," Journal of the Academy of Marketing Science, Springer, vol. 52(4), pages 1216-1239, July.
    12. Burbano, Vanessa & Padilla, Nicolas & Meier, Stephan, 2020. "Gender Differences in Preferences for Meaning at Work," IZA Discussion Papers 13053, Institute of Labor Economics (IZA).
    13. Robert Kubinec & Haillie Na‐Kyung Lee & Andrey Tomashevskiy, 2021. "Politically connected companies are less likely to shutdown due to COVID‐19 restrictions," Social Science Quarterly, Southwestern Social Science Association, vol. 102(5), pages 2155-2169, September.
    14. Barrington-Leigh, C.P., 2024. "The econometrics of happiness: Are we underestimating the returns to education and income?," Journal of Public Economics, Elsevier, vol. 230(C).
    15. Salvatore Nunnari & Massimiliano Pozzi, 2022. "Meta-Analysis of Inequality Aversion Estimates," CESifo Working Paper Series 9851, CESifo.
    16. Zhou, Bin & Xie, Jia-Rong & Yan, Xiao-Yong & Wang, Nianxin & Wang, Bing-Hong, 2017. "A model of task-deletion mechanism based on the priority queueing system of Barabási," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 466(C), pages 415-421.
    17. Chen, Ning & Zhu, Xuzhen & Chen, Yanyan, 2019. "Information spreading on complex networks with general group distribution," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 523(C), pages 671-676.
    18. Andreas Kryger Jensen & Claus Thorn Ekstrøm, 2021. "Quantifying the trendiness of trends," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(1), pages 98-121, January.
    19. Lauderdale, Benjamin E. & Bailey, Delia & Blumenau, Jack & Rivers, Douglas, 2020. "Model-based pre-election polling for national and sub-national outcomes in the US and UK," International Journal of Forecasting, Elsevier, vol. 36(2), pages 399-413.
    20. Zhenpeng Li & Xijin Tang & Zhenjie Hong, 2022. "Collective attention dynamic induced by novelty decay," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 95(8), pages 1-11, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0242201. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.