IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v7y2016i1d10.1038_ncomms12438.html
   My bibliography  Save this article

Gaze data reveal distinct choice processes underlying model-based and model-free reinforcement learning

Author

Listed:
  • Arkady Konovalov

    (The Ohio State University)

  • Ian Krajbich

    (The Ohio State University
    The Ohio State University)

Abstract

Organisms appear to learn and make decisions using different strategies known as model-free and model-based learning; the former is mere reinforcement of previously rewarded actions and the latter is a forward-looking strategy that involves evaluation of action-state transition probabilities. Prior work has used neural data to argue that both model-based and model-free learners implement a value comparison process at trial onset, but model-based learners assign more weight to forward-looking computations. Here using eye-tracking, we report evidence for a different interpretation of prior results: model-based subjects make their choices prior to trial onset. In contrast, model-free subjects tend to ignore model-based aspects of the task and instead seem to treat the decision problem as a simple comparison process between two differentially valued items, consistent with previous work on sequential-sampling models of decision making. These findings illustrate a problem with assuming that experimental subjects make their decisions at the same prescribed time.

Suggested Citation

  • Arkady Konovalov & Ian Krajbich, 2016. "Gaze data reveal distinct choice processes underlying model-based and model-free reinforcement learning," Nature Communications, Nature, vol. 7(1), pages 1-11, November.
  • Handle: RePEc:nat:natcom:v:7:y:2016:i:1:d:10.1038_ncomms12438
    DOI: 10.1038/ncomms12438
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/ncomms12438
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/ncomms12438?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Johann Lussange & Ivan Lazarevich & Sacha Bourgeois-Gironde & Stefano Palminteri & Boris Gutkin, 2021. "Modelling Stock Markets by Multi-agent Reinforcement Learning," Computational Economics, Springer;Society for Computational Economics, vol. 57(1), pages 113-147, January.
    2. repec:cup:judgdm:v:14:y:2019:i:4:p:381-394 is not listed on IDEAS
    3. Arkady Konovalov & Ian Krajbich, 2019. "Revealed strength of preference: Inference from response times," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 14(4), pages 381-394, July.
    4. Geoffrey Fisher, 2023. "Measuring the Factors Influencing Purchasing Decisions: Evidence From Cursor Tracking and Cognitive Modeling," Management Science, INFORMS, vol. 69(8), pages 4558-4578, August.
    5. Fisher, Geoffrey, 2021. "A multiattribute attentional drift diffusion model," Organizational Behavior and Human Decision Processes, Elsevier, vol. 165(C), pages 167-182.
    6. Tarikere T. Niranjan & Narendra K. Ghosalya & Srinagesh Gavirneni, 2022. "Crying Wolf and a Knowing Wink: A Behavioral Study of Order Inflation and Discounting in Supply Chains," Production and Operations Management, Production and Operations Management Society, vol. 31(3), pages 1071-1088, March.
    7. Molter, Felix & Thomas, Armin W. & Heekeren, Hauke R. & Mohr, Peter N. C., 2019. "GLAMbox: A Python toolbox for investigating the association between gaze allocation and decision behaviour," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 14(12), pages 1-23.
    8. Amasino, Dianna R. & Dolgin, Jack & Huettel, Scott A., 2023. "Eyes on the account size: Interactions between attention and budget in consumer choice," Journal of Economic Psychology, Elsevier, vol. 97(C).
    9. Felix Molter & Armin W Thomas & Hauke R Heekeren & Peter N C Mohr, 2019. "GLAMbox: A Python toolbox for investigating the association between gaze allocation and decision behaviour," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-23, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:7:y:2016:i:1:d:10.1038_ncomms12438. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.