IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2409.05798.html
   My bibliography  Save this paper

Enhancing Preference-based Linear Bandits via Human Response Time

Author

Listed:
  • Shen Li
  • Yuyang Zhang
  • Zhaolin Ren
  • Claire Liang
  • Na Li
  • Julie A. Shah

Abstract

Interactive preference learning systems present humans with queries as pairs of options; humans then select their preferred choice, allowing the system to infer preferences from these binary choices. While binary choice feedback is simple and widely used, it offers limited information about preference strength. To address this, we leverage human response times, which inversely correlate with preference strength, as complementary information. We introduce a computationally efficient method based on the EZ-diffusion model, combining choices and response times to estimate the underlying human utility function. Theoretical and empirical comparisons with traditional choice-only estimators show that for queries where humans have strong preferences (i.e., "easy" queries), response times provide valuable complementary information and enhance utility estimates. We integrate this estimator into preference-based linear bandits for fixed-budget best-arm identification. Simulations on three real-world datasets demonstrate that incorporating response times significantly accelerates preference learning.

Suggested Citation

  • Shen Li & Yuyang Zhang & Zhaolin Ren & Claire Liang & Na Li & Julie A. Shah, 2024. "Enhancing Preference-based Linear Bandits via Human Response Time," Papers 2409.05798, arXiv.org, revised Oct 2024.
  • Handle: RePEc:arx:papers:2409.05798
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2409.05798
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Clithero, John A., 2018. "Improving out-of-sample predictions using response times and a model of the decision process," Journal of Economic Behavior & Organization, Elsevier, vol. 148(C), pages 344-375.
    2. Carlos Alós-Ferrer & Ernst Fehr & Nick Netzer, 2021. "Time Will Tell: Recovering Preferences When Choices Are Noisy," Journal of Political Economy, University of Chicago Press, vol. 129(6), pages 1828-1877.
    3. Thomas, Armin W. & Molter, Felix & Krajbich, Ian & Heekeren, Hauke R. & Mohr, Peter N. C., 2019. "Gaze bias differences capture individual choice behaviour," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 3(6), pages 625-635.
    4. Carlo Baldassi & Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci & Marco Pirazzini, 2020. "A Behavioral Characterization of the Drift Diffusion Model and Its Multialternative Extension for Choice Under Time Pressure," Management Science, INFORMS, vol. 66(11), pages 5075-5093, November.
    5. Armin W. Thomas & Felix Molter & Ian Krajbich & Hauke R. Heekeren & Peter N. C. Mohr, 2019. "Gaze bias differences capture individual choice behaviour," Nature Human Behaviour, Nature, vol. 3(6), pages 625-635, June.
    6. Paat Rusmevichientong & David Shmoys & Chaoxu Tong & Huseyin Topaloglu, 2014. "Assortment Optimization under the Multinomial Logit Model with Random Choice Parameters," Production and Operations Management, Production and Operations Management Society, vol. 23(11), pages 2023-2039, November.
    7. Shipra Agrawal & Vashist Avadhanula & Vineet Goyal & Assaf Zeevi, 2019. "MNL-Bandit: A Dynamic Learning Approach to Assortment Selection," Operations Research, INFORMS, vol. 67(5), pages 1453-1485, September.
    8. Clithero, John A., 2018. "Response times in economics: Looking through the lens of sequential sampling models," Journal of Economic Psychology, Elsevier, vol. 69(C), pages 61-86.
    9. Ryan Webb, 2019. "The (Neural) Dynamics of Stochastic Choice," Management Science, INFORMS, vol. 65(1), pages 230-255, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carlo Baldassi & Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci & Marco Pirazzini, 2020. "A Behavioral Characterization of the Drift Diffusion Model and Its Multialternative Extension for Choice Under Time Pressure," Management Science, INFORMS, vol. 66(11), pages 5075-5093, November.
    2. S. Cerreia-Vioglio & F. Maccheroni & M. Marinacci & A. Rustichini, 2017. "Multinomial logit processes and preference discovery: inside and outside the black box," Working Papers 615, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    3. Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci, 2020. "Multinomial logit processes and preference discovery: outside and inside the black box," Working Papers 663, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    4. Bansal, Prateek & Kim, Eui-Jin & Ozdemir, Semra, 2024. "Discrete choice experiments with eye-tracking: How far we have come and ways forward," Journal of choice modelling, Elsevier, vol. 51(C).
    5. Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci & Aldo Rustichini, 2023. "Multinomial Logit Processes and Preference Discovery: Inside and Outside the Black Box," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(3), pages 1155-1194.
    6. Shuo Liu & Nick Netzer, 2023. "Happy Times: Measuring Happiness Using Response Times," American Economic Review, American Economic Association, vol. 113(12), pages 3289-3322, December.
    7. Carlos Alós-Ferrer & Michele Garagnani, 2022. "Strength of preference and decisions under risk," Journal of Risk and Uncertainty, Springer, vol. 64(3), pages 309-329, June.
    8. Duffy, Sean & Smith, John, 2020. "An economist and a psychologist form a line: What can imperfect perception of length tell us about stochastic choice?," MPRA Paper 99417, University Library of Munich, Germany.
    9. Moshe Glickman & Orian Sharoni & Dino J Levy & Ernst Niebur & Veit Stuphorn & Marius Usher, 2019. "The formation of preference in risky choice," PLOS Computational Biology, Public Library of Science, vol. 15(8), pages 1-25, August.
    10. Hébert, Benjamin & Woodford, Michael, 2023. "Rational inattention when decisions take time," Journal of Economic Theory, Elsevier, vol. 208(C).
    11. Konrad Grabiszewski & Alex Horenstein, 2022. "Profiling dynamic decision-makers," PLOS ONE, Public Library of Science, vol. 17(4), pages 1-22, April.
    12. Cary Frydman & Ian Krajbich, 2022. "Using Response Times to Infer Others’ Private Information: An Application to Information Cascades," Management Science, INFORMS, vol. 68(4), pages 2970-2986, April.
    13. Fischbacher, Urs & Hausfeld, Jan & Renerte, Baiba, 2022. "Strategic incentives undermine gaze as a signal of prosocial motives," Games and Economic Behavior, Elsevier, vol. 136(C), pages 63-91.
    14. Aleksandr Alekseev, 2019. "Using response times to measure ability on a cognitive task," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 65-75, August.
    15. Frederick Callaway & Antonio Rangel & Thomas L Griffiths, 2021. "Fixation patterns in simple choice reflect optimal information sampling," PLOS Computational Biology, Public Library of Science, vol. 17(3), pages 1-29, March.
    16. Carlos Alós-Ferrer & Ernst Fehr & Nick Netzer, 2021. "Time Will Tell: Recovering Preferences When Choices Are Noisy," Journal of Political Economy, University of Chicago Press, vol. 129(6), pages 1828-1877.
    17. Carlos Alós-Ferrer & Johannes Buckenmaier, 2021. "Cognitive sophistication and deliberation times," Experimental Economics, Springer;Economic Science Association, vol. 24(2), pages 558-592, June.
    18. Andrew Schotter & Isabel Trevino, 2021. "Is response time predictive of choice? An experimental study of threshold strategies," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 87-117, March.
    19. Nathan Kallus & Madeleine Udell, 2020. "Dynamic Assortment Personalization in High Dimensions," Operations Research, INFORMS, vol. 68(4), pages 1020-1037, July.
    20. Kris Johnson Ferreira & Joel Goh, 2021. "Assortment Rotation and the Value of Concealment," Management Science, INFORMS, vol. 67(3), pages 1489-1507, March.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2409.05798. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.