IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0094308.html
   My bibliography  Save this article

Sequential Decisions: A Computational Comparison of Observational and Reinforcement Accounts

Author

Listed:
  • Nazanin Mohammadi Sepahvand
  • Elisabeth Stöttinger
  • James Danckert
  • Britt Anderson

Abstract

Right brain damaged patients show impairments in sequential decision making tasks for which healthy people do not show any difficulty. We hypothesized that this difficulty could be due to the failure of right brain damage patients to develop well-matched models of the world. Our motivation is the idea that to navigate uncertainty, humans use models of the world to direct the decisions they make when interacting with their environment. The better the model is, the better their decisions are. To explore the model building and updating process in humans and the basis for impairment after brain injury, we used a computational model of non-stationary sequence learning. RELPH (Reinforcement and Entropy Learned Pruned Hypothesis space) was able to qualitatively and quantitatively reproduce the results of left and right brain damaged patient groups and healthy controls playing a sequential version of Rock, Paper, Scissors. Our results suggests that, in general, humans employ a sub-optimal reinforcement based learning method rather than an objectively better statistical learning approach, and that differences between right brain damaged and healthy control groups can be explained by different exploration policies, rather than qualitatively different learning mechanisms.

Suggested Citation

  • Nazanin Mohammadi Sepahvand & Elisabeth Stöttinger & James Danckert & Britt Anderson, 2014. "Sequential Decisions: A Computational Comparison of Observational and Reinforcement Accounts," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-8, April.
  • Handle: RePEc:plo:pone00:0094308
    DOI: 10.1371/journal.pone.0094308
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0094308
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0094308&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0094308?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. James G. March, 1991. "Exploration and Exploitation in Organizational Learning," Organization Science, INFORMS, vol. 2(1), pages 71-87, February.
    2. repec:bla:jecsur:v:14:y:2000:i:1:p:101-18 is not listed on IDEAS
    3. Nathaniel D. Daw & John P. O'Doherty & Peter Dayan & Ben Seymour & Raymond J. Dolan, 2006. "Cortical substrates for exploratory decisions in humans," Nature, Nature, vol. 441(7095), pages 876-879, June.
    4. Elise Payzan-LeNestour & Peter Bossaerts, 2011. "Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings," PLOS Computational Biology, Public Library of Science, vol. 7(1), pages 1-14, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Erik Brockbank & Edward Vul, 2021. "Formalizing Opponent Modeling with the Rock, Paper, Scissors Game," Games, MDPI, vol. 12(3), pages 1-20, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter S. Riefer & Bradley C. Love, 2015. "Unfazed by Both the Bull and Bear: Strategic Exploration in Dynamic Environments," Games, MDPI, vol. 6(3), pages 1-11, August.
    2. Christina Fang & Daniel Levinthal, 2009. "Near-Term Liability of Exploitation: Exploration and Exploitation in Multistage Problems," Organization Science, INFORMS, vol. 20(3), pages 538-551, June.
    3. Hu, Yingyao & Kayaba, Yutaka & Shum, Matthew, 2013. "Nonparametric learning rules from bandit experiments: The eyes have it!," Games and Economic Behavior, Elsevier, vol. 81(C), pages 215-231.
    4. Oded Berger-Tal & Jonathan Nathan & Ehud Meron & David Saltz, 2014. "The Exploration-Exploitation Dilemma: A Multidisciplinary Framework," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-8, April.
    5. Phanish Puranam & Murali Swamy, 2016. "How Initial Representations Shape Coupled Learning Processes," Organization Science, INFORMS, vol. 27(2), pages 323-335, April.
    6. Michelle Rogan & Marie Louise Mors, 2014. "A Network Perspective on Individual-Level Ambidexterity in Organizations," Organization Science, INFORMS, vol. 25(6), pages 1860-1877, December.
    7. Bruno B Averbeck, 2015. "Theory of Choice in Bandit, Information Sampling and Foraging Tasks," PLOS Computational Biology, Public Library of Science, vol. 11(3), pages 1-28, March.
    8. De Gennaro Aquino, Luca & Sornette, Didier & Strub, Moris S., 2023. "Portfolio selection with exploration of new investment assets," European Journal of Operational Research, Elsevier, vol. 310(2), pages 773-792.
    9. Alina Ferecatu & Arnaud De Bruyn, 2022. "Understanding Managers’ Trade-Offs Between Exploration and Exploitation," Marketing Science, INFORMS, vol. 41(1), pages 139-165, January.
    10. Arman Avadikyan & Gilles Lambert & Christophe Lerch, 2016. "A Multi-Level Perspective on Ambidexterity: The Case of a Synchrotron Research Facility," Working Papers of BETA 2016-44, Bureau d'Economie Théorique et Appliquée, UDS, Strasbourg.
    11. Henri A. Schildt & Markku V.J. Maula & Thomas Keil, 2005. "Explorative and Exploitative Learning from External Corporate Ventures," Entrepreneurship Theory and Practice, , vol. 29(4), pages 493-515, July.
    12. Giuliani, Elisa & Martinelli, Arianna & Rabellotti, Roberta, 2016. "Is Co-Invention Expediting Technological Catch Up? A Study of Collaboration between Emerging Country Firms and EU Inventors," World Development, Elsevier, vol. 77(C), pages 192-205.
    13. Insoo Cho & Peter F. Orazem, 2021. "How endogenous risk preferences and sample selection affect analysis of firm survival," Small Business Economics, Springer, vol. 56(4), pages 1309-1332, April.
    14. Tomasz Helbin & Amy Van Looy, 2021. "Is Business Process Management (BPM) Ready for Ambidexterity? Conceptualization, Implementation Guidelines and Research Agenda," Sustainability, MDPI, vol. 13(4), pages 1-25, February.
    15. Son K. Lam & Thomas E. DeCarlo & Ashish Sharma, 2019. "Salesperson ambidexterity in customer engagement: do customer base characteristics matter?," Journal of the Academy of Marketing Science, Springer, vol. 47(4), pages 659-680, July.
    16. Jonathan H. Reed, 2022. "Operational and strategic change during temporary turbulence: evidence from the COVID-19 pandemic," Operations Management Research, Springer, vol. 15(1), pages 589-608, June.
    17. Alan Hevner & Isabelle Comyn-Wattiau & Jacky Akoka & Nicolas Prat, 2018. "A pragmatic approach for identifying and managing design science research goals and evaluation criteria," Post-Print hal-02283783, HAL.
    18. Felipe A. Csaszar & Nicolaj Siggelkow, 2010. "How Much to Copy? Determinants of Effective Imitation Breadth," Organization Science, INFORMS, vol. 21(3), pages 661-676, June.
    19. Bruneel, Johan & Clarysse, Bart & Bobelyn, Annelies & Wright, Mike, 2020. "Liquidity events and VC-backed academic spin-offs: The role of search alliances," Research Policy, Elsevier, vol. 49(10).
    20. Sylvie Héroux & Mélanie Roussy, 2020. "Three cases of compliance with governance regulation: an organizational learning perspective," Journal of Management & Governance, Springer;Accademia Italiana di Economia Aziendale (AIDEA), vol. 24(2), pages 449-479, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0094308. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.