IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1006803.html
   My bibliography  Save this article

Improving the reliability of model-based decision-making estimates in the two-stage decision task with reaction-times and drift-diffusion modeling

Author

Listed:
  • Nitzan Shahar
  • Tobias U Hauser
  • Michael Moutoussis
  • Rani Moran
  • Mehdi Keramati
  • NSPN consortium
  • Raymond J Dolan

Abstract

A well-established notion in cognitive neuroscience proposes that multiple brain systems contribute to choice behaviour. These include: (1) a model-free system that uses values cached from the outcome history of alternative actions, and (2) a model-based system that considers action outcomes and the transition structure of the environment. The widespread use of this distinction, across a range of applications, renders it important to index their distinct influences with high reliability. Here we consider the two-stage task, widely considered as a gold standard measure for the contribution of model-based and model-free systems to human choice. We tested the internal/temporal stability of measures from this task, including those estimated via an established computational model, as well as an extended model using drift-diffusion. Drift-diffusion modeling suggested that both choice in the first stage, and RTs in the second stage, are directly affected by a model-based/free trade-off parameter. Both parameter recovery and the stability of model-based estimates were poor but improved substantially when both choice and RT were used (compared to choice only), and when more trials (than conventionally used in research practice) were included in our analysis. The findings have implications for interpretation of past and future studies based on the use of the two-stage task, as well as for characterising the contribution of model-based processes to choice behaviour.Author summary: In this paper, we report a reliability analysis for the estimation of “model basedness”—a psychological construct that informs a wealth of studies in animal, human and clinical research. We consider an exemplar paradigm, the two-step task, widely used in the recent literature. We report low reliability for model-agnostic model-based estimates, as well as computational model parameter estimates. We suggest how a model-based/free trade-off might affect reaction-time variability in this task, and go on to suggest use of model parameter estimates based on a combination of choice and RT. Finally, we demonstrate that combining choice and RT estimates improves both model-agnostic and algorithmic model-based estimates.

Suggested Citation

  • Nitzan Shahar & Tobias U Hauser & Michael Moutoussis & Rani Moran & Mehdi Keramati & NSPN consortium & Raymond J Dolan, 2019. "Improving the reliability of model-based decision-making estimates in the two-stage decision task with reaction-times and drift-diffusion modeling," PLOS Computational Biology, Public Library of Science, vol. 15(2), pages 1-25, February.
    • Handle: RePEc:plo:pcbi00:1006803
    DOI: 10.1371/journal.pcbi.1006803
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006803
    Download Restriction: no
    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1006803&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pcbi.1006803?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Amir Dezfouli & Bernard W Balleine, 2013. "Actions, Action Sequences and Habits: Evidence That Goal-Directed and Habitual Action Control Are Hierarchically Organized," PLOS Computational Biology, Public Library of Science, vol. 9(12), pages 1-14, December.
    2. Ian Krajbich & Björn Bartling & Todd Hare & Ernst Fehr, 2015. "Rethinking fast and slow based on a critique of reaction-time reverse inference," Nature Communications, Nature, vol. 6(1), pages 1-9, November.
    3. Carolina Feher da Silva & Todd A Hare, 2018. "A note on the analysis of two-stage task results: How changes in task structure affect what model-free and model-based strategies predict about the effects of reward and transition on the stay probabi," PLOS ONE, Public Library of Science, vol. 13(4), pages 1-13, April.
    4. Christina M. Gremel & Rui M. Costa, 2013. "Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions," Nature Communications, Nature, vol. 4(1), pages 1-12, October.
    5. Wouter Kool & Fiery A Cushman & Samuel J Gershman, 2016. "When Does Model-Based Control Pay Off?," PLOS Computational Biology, Public Library of Science, vol. 12(8), pages 1-34, August.
    6. G. N. Wilkinson & C. E. Rogers, 1973. "Symbolic Description of Factorial Models for Analysis of Variance," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 22(3), pages 392-399, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Karima Chakroun & Antonius Wiehler & Ben Wagner & David Mathar & Florian Ganzer & Thilo Eimeren & Tobias Sommer & Jan Peters, 2023. "Dopamine regulates decision thresholds in human reinforcement learning in males," Nature Communications, Nature, vol. 14(1), pages 1-14, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Thomas Akam & Rui Costa & Peter Dayan, 2015. "Simple Plans or Sophisticated Habits? State, Transition and Learning Interactions in the Two-Step Task," PLOS Computational Biology, Public Library of Science, vol. 11(12), pages 1-25, December.
    2. Bruno Miranda & W M Nishantha Malalasekera & Timothy E Behrens & Peter Dayan & Steven W Kennerley, 2020. "Combined model-free and model-sensitive reinforcement learning in non-human primates," PLOS Computational Biology, Public Library of Science, vol. 16(6), pages 1-25, June.
    3. Carolina Feher da Silva & Todd A Hare, 2018. "A note on the analysis of two-stage task results: How changes in task structure affect what model-free and model-based strategies predict about the effects of reward and transition on the stay probabi," PLOS ONE, Public Library of Science, vol. 13(4), pages 1-13, April.
    4. Jacquemet, N. & Luchini, S. & Malézieux, A. & Shogren, J.F., 2020. "Who’ll stop lying under oath? Empirical evidence from tax evasion games," European Economic Review, Elsevier, vol. 124(C).
    5. Nicolas Jacquemet & Stephane Luchini & Jason Shogren & Verity Watson, 2019. "Discrete Choice under Oaths," Post-Print halshs-02136103, HAL.
    6. Payne, Roger W., 1998. "Design keys, pseudo-factors and general balance," Computational Statistics & Data Analysis, Elsevier, vol. 29(2), pages 217-229, December.
    7. Myrseth, Kristian Ove R. & Wollbrant, Conny E., 2017. "Cognitive foundations of cooperation revisited: Commentary on Rand et al. (2012, 2014)," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 69(C), pages 133-138.
    8. Castillo, Marco & Dickinson, David L., 2022. "Sleep restriction increases coordination failure," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 358-370.
    9. Lohse, Tim & Simon, Sven A. & Konrad, Kai A., 2018. "Deception under time pressure: Conscious decision or a problem of awareness?," Journal of Economic Behavior & Organization, Elsevier, vol. 146(C), pages 31-42.
    10. Alós-Ferrer, Carlos & Garagnani, Michele, 2020. "The cognitive foundations of cooperation," Journal of Economic Behavior & Organization, Elsevier, vol. 175(C), pages 71-85.
    11. Quirin Gehmacher & Juliane Schubert & Fabian Schmidt & Thomas Hartmann & Patrick Reisinger & Sebastian Rösch & Konrad Schwarz & Tzvetan Popov & Maria Chait & Nathan Weisz, 2024. "Eye movements track prioritized auditory features in selective attention to natural speech," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    12. Carlos Alós-Ferrer & Johannes Buckenmaier, 2021. "Cognitive sophistication and deliberation times," Experimental Economics, Springer;Economic Science Association, vol. 24(2), pages 558-592, June.
    13. Drew Fudenberg & Whitney Newey & Philipp Strack & Tomasz Strzalecki, 2020. "Testing the drift-diffusion model," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 117(52), pages 33141-33148, December.
    14. Francesco Paolo Ulloa Severino & Oluwadamilola O. Lawal & Kristina Sakers & Shiyi Wang & Namsoo Kim & Alexander David Friedman & Sarah Anne Johnson & Chaichontat Sriworarat & Ryan H. Hughes & Scott H., 2023. "Training-induced circuit-specific excitatory synaptogenesis in mice is required for effort control," Nature Communications, Nature, vol. 14(1), pages 1-22, December.
    15. Barrafrem, Kinga & Hausfeld, Jan, 2020. "Tracing risky decisions for oneself and others: The role of intuition and deliberation," Journal of Economic Psychology, Elsevier, vol. 77(C).
    16. Jérôme Hergueux & Nicolas Jacquemet & Stéphane Luchini & Jason F. Shogren, 2022. "Leveraging the Honor Code: Public Goods Contributions under Oath," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 81(3), pages 591-616, March.
    17. Amir Dezfouli & Bernard W Balleine, 2019. "Learning the structure of the world: The adaptive nature of state-space and action representations in multi-stage decision-making," PLOS Computational Biology, Public Library of Science, vol. 15(9), pages 1-22, September.
    18. Exley, Christine L. & Petrie, Ragan, 2018. "The impact of a surprise donation ask," Journal of Public Economics, Elsevier, vol. 158(C), pages 152-167.
    19. repec:cup:judgdm:v:16:y:2021:i:4:p:844-897 is not listed on IDEAS
    20. Sravani Kondapavulur & Stefan M. Lemke & David Darevsky & Ling Guo & Preeya Khanna & Karunesh Ganguly, 2022. "Transition from predictable to variable motor cortex and striatal ensemble patterning during behavioral exploration," Nature Communications, Nature, vol. 13(1), pages 1-17, December.
    21. Andres Montealegre & William Jimenez-Leal, 2019. "The role of trust in the social heuristics hypothesis," PLOS ONE, Public Library of Science, vol. 14(5), pages 1-16, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1006803. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.