IDEAS home Printed from https://ideas.repec.org/a/eee/jeborg/v148y2018icp344-375.html
   My bibliography  Save this article

Improving out-of-sample predictions using response times and a model of the decision process

Author

Listed:
  • Clithero, John A.

Abstract

A basic problem in empirical economics involves using data from one domain to make out-of-sample predictions for a different, but related environment. When the choice data are binary, a canonical method for making these types of predictions is the logistic choice model. This paper investigates whether it is possible to improve out-of-sample predictions by changing two aspects of the canonical approach: 1) Using response times in addition to the choice data, and 2) Combining them using a model from the psychology and neuroscience literature, the Drift-Diffusion Model (DDM). Two experiments compare the out-of-sample choice prediction accuracies of both methods and in both cases the DDM method outperforms a logistic prediction method. Furthermore, the DDM allows for out-of-sample process predictions. Both experiments validate the DDM as a method for predicting out-of-sample response times.

Suggested Citation

  • Clithero, John A., 2018. "Improving out-of-sample predictions using response times and a model of the decision process," Journal of Economic Behavior & Organization, Elsevier, vol. 148(C), pages 344-375.
  • Handle: RePEc:eee:jeborg:v:148:y:2018:i:c:p:344-375
    DOI: 10.1016/j.jebo.2018.02.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167268118300398
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jebo.2018.02.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Patil, Anand & Huard, David & Fonnesbeck, Christopher J., 2010. "PyMC: Bayesian Stochastic Modelling in Python," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 35(i04).
    2. Jörg Rieskamp & Jerome R. Busemeyer & Barbara A. Mellers, 2006. "Extending the Bounds of Rationality: Evidence and Theories of Preferential Choice," Journal of Economic Literature, American Economic Association, vol. 44(3), pages 631-661, September.
    3. Marina Agranov & Andrew Caplin & Chloe Tergiman, 2015. "Naive play and the process of choice in guessing games," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(2), pages 146-157, December.
    4. Filip Matêjka & Alisdair McKay, 2015. "Rational Inattention to Discrete Choices: A New Foundation for the Multinomial Logit Model," American Economic Review, American Economic Association, vol. 105(1), pages 272-298, January.
    5. Peter Moffatt, 2005. "Stochastic Choice and the Allocation of Cognitive Effort," Experimental Economics, Springer;Economic Science Association, vol. 8(4), pages 369-388, December.
    6. Justin Wolfers & Eric Zitzewitz, 2004. "Prediction Markets," Journal of Economic Perspectives, American Economic Association, vol. 18(2), pages 107-126, Spring.
    7. Pedro Bordalo & Nicola Gennaioli & Andrei Shleifer, 2013. "Salience and Consumer Choice," Journal of Political Economy, University of Chicago Press, vol. 121(5), pages 803-843.
    8. Goeree, Jacob K. & Holt, Charles A., 2004. "A model of noisy introspection," Games and Economic Behavior, Elsevier, vol. 46(2), pages 365-382, February.
    9. Aldo Rustichini, 2009. "Is There a Method of Neuroeconomics?," American Economic Journal: Microeconomics, American Economic Association, vol. 1(2), pages 48-59, August.
    10. Elena Reutskaja & Rosemarie Nagel & Colin F. Camerer & Antonio Rangel, 2011. "Search Dynamics in Consumer Choice under Time Pressure: An Eye-Tracking Study," American Economic Review, American Economic Association, vol. 101(2), pages 900-926, April.
    11. Isabelle Brocas & Juan D. Carrillo & Stephanie W. Wang & Colin F. Camerer, 2014. "Imperfect Choice or Imperfect Attention? Understanding Strategic Thinking in Private Information Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(3), pages 944-970.
    12. Ian Krajbich & Todd Hare & Björn Bartling & Yosuke Morishima & Ernst Fehr, 2015. "A Common Mechanism Underlying Food Choice and Social Decisions," PLOS Computational Biology, Public Library of Science, vol. 11(10), pages 1-24, October.
    13. Basten U & Biele G. P. & Heekeren H. R. & Fiebach, 2010. "How the brain integrates costs and benefits during decision making," SFB 649 Discussion Papers SFB649DP2010-063, Sonderforschungsbereich 649, Humboldt University, Berlin, Germany.
    14. Ariel Rubinstein, 2007. "Instinctive and Cognitive Reasoning: A Study of Response Times," Economic Journal, Royal Economic Society, vol. 117(523), pages 1243-1259, October.
    15. Faruk Gul & Paulo Natenzon & Wolfgang Pesendorfer, 2014. "Random Choice as Behavioral Optimization," Econometrica, Econometric Society, vol. 82, pages 1873-1912, September.
    16. Clément Levallois & John A. Clithero & Paul Wouters & Ale Smidts & Scott A. Huettel, 2012. "Translating upwards : linking the neural and social sciences via neuroeconomics," Post-Print hal-02313160, HAL.
    17. Ernst Fehr & Antonio Rangel, 2011. "Neuroeconomic Foundations of Economic Choice--Recent Advances," Journal of Economic Perspectives, American Economic Association, vol. 25(4), pages 3-30, Fall.
    18. Christian A Rodriguez & Brandon M Turner & Samuel M McClure, 2014. "Intertemporal Choice as Discounted Value Accumulation," PLOS ONE, Public Library of Science, vol. 9(2), pages 1-9, February.
    19. Alec Smith & B. Douglas Bernheim & Colin F. Camerer & Antonio Rangel, 2014. "Neural Activity Reveals Preferences without Choices," American Economic Journal: Microeconomics, American Economic Association, vol. 6(2), pages 1-36, May.
    20. repec:cup:judgdm:v:5:y:2010:i:6:p:437-449 is not listed on IDEAS
    21. Joseph Tao-yi Wang & Michael Spezio & Colin F. Camerer, 2010. "Pinocchio's Pupil: Using Eyetracking and Pupil Dilation to Understand Truth Telling and Deception in Sender-Receiver Games," American Economic Review, American Economic Association, vol. 100(3), pages 984-1007, June.
    22. H. R. Heekeren & S. Marrett & P. A. Bandettini & L. G. Ungerleider, 2004. "A general mechanism for perceptual decision-making in the human brain," Nature, Nature, vol. 431(7010), pages 859-862, October.
    23. Hyunyoung Choi & Hal Varian, 2012. "Predicting the Present with Google Trends," The Economic Record, The Economic Society of Australia, vol. 88(s1), pages 2-9, June.
    24. B. Douglas Bernheim, 2009. "On the Potential of Neuroeconomics: A Critical (but Hopeful) Appraisal," American Economic Journal: Microeconomics, American Economic Association, vol. 1(2), pages 1-41, August.
    25. Brocas, Isabelle, 2012. "Information processing and decision-making: Evidence from the brain sciences and implications for economics," Journal of Economic Behavior & Organization, Elsevier, vol. 83(3), pages 292-310.
    26. David J. Spiegelhalter & Nicola G. Best & Bradley P. Carlin & Angelika Van Der Linde, 2002. "Bayesian measures of model complexity and fit," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(4), pages 583-639, October.
    27. Thomas Otter & Joe Johnson & Jörg Rieskamp & Greg Allenby & Jeff Brazell & Adele Diederich & J. Hutchinson & Steven MacEachern & Shiling Ruan & Jim Townsend, 2008. "Sequential sampling models of choice: Some recent advances," Marketing Letters, Springer, vol. 19(3), pages 255-267, December.
    28. Ariel Rubinstein, 2016. "A Typology of Players: Between Instinctive and Contemplative," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(2), pages 859-890.
    29. Mervyn Stone, 1960. "Models for choice-reaction time," Psychometrika, Springer;The Psychometric Society, vol. 25(3), pages 251-260, September.
    30. Ian Krajbich & Bastiaan Oud & Ernst Fehr, 2014. "Benefits of Neuroeconomic Modeling: New Policy Interventions and Predictors of Preference," American Economic Review, American Economic Association, vol. 104(5), pages 501-506, May.
    31. Wilcox, Nathaniel T, 1993. "Lottery Choice: Incentives, Complexity and Decision Time," Economic Journal, Royal Economic Society, vol. 103(421), pages 1397-1417, November.
    32. Komlos, John & Kelly, Inas (ed.), 2016. "The Oxford Handbook of Economics and Human Biology," OUP Catalogue, Oxford University Press, number 9780199389292.
    33. Colin F. Camerer, 2013. "Goals, Methods, and Progress in Neuroeconomics," Annual Review of Economics, Annual Reviews, vol. 5(1), pages 425-455, May.
    34. Ariel Rubinstein, 2007. "Instinctive and Cognitive Reasoning: Response Times Study," Levine's Bibliography 321307000000001011, UCLA Department of Economics.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Clithero, John A., 2018. "Response times in economics: Looking through the lens of sequential sampling models," Journal of Economic Psychology, Elsevier, vol. 69(C), pages 61-86.
    2. Guidon Fenig & Giovanni Gallipoli & Yoram Halevy, 2018. "Piercing the 'Payoff Function' Veil: Tracing Beliefs and Motives," Working Papers tecipa-619, University of Toronto, Department of Economics.
    3. Arkady Konovalov & Ian Krajbich, 2016. "Revealed Indifference: Using Response Times to Infer Preferences," Working Papers 16-01, Ohio State University, Department of Economics.
    4. Recalde, María P. & Riedl, Arno & Vesterlund, Lise, 2018. "Error-prone inference from response time: The case of intuitive generosity in public-good games," Journal of Public Economics, Elsevier, vol. 160(C), pages 132-147.
    5. Fadong Chen & Urs Fischbacher, 2016. "Response time and click position: cheap indicators of preferences," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 2(2), pages 109-126, November.
    6. Carlos Alós-Ferrer & Johannes Buckenmaier, 2021. "Cognitive sophistication and deliberation times," Experimental Economics, Springer;Economic Science Association, vol. 24(2), pages 558-592, June.
    7. Jan Hausfeld & Sven Resnjanskij, 2017. "Risky Decisions and the Opportunity Costs of Time," TWI Research Paper Series 108, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    8. Daniel Navarro-Martinez & Graham Loomes & Andrea Isoni & David Butler & Larbi Alaoui, 2018. "Boundedly rational expected utility theory," Journal of Risk and Uncertainty, Springer, vol. 57(3), pages 199-223, December.
    9. Grabiszewski, Konrad & Horenstein, Alex, 2020. "Effort is not a monotonic function of skills: Results from a global mobile experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 176(C), pages 634-652.
    10. Carlos Alós-Ferrer & Ernst Fehr & Nick Netzer, 2021. "Time Will Tell: Recovering Preferences When Choices Are Noisy," Journal of Political Economy, University of Chicago Press, vol. 129(6), pages 1828-1877.
    11. Arkady Konovalov & Ian Krajbich, 2019. "Revealed strength of preference: Inference from response times," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 14(4), pages 381-394, July.
    12. Ryan Webb, 2019. "The (Neural) Dynamics of Stochastic Choice," Management Science, INFORMS, vol. 65(1), pages 230-255, January.
    13. Cary Frydman & Ian Krajbich, 2022. "Using Response Times to Infer Others’ Private Information: An Application to Information Cascades," Management Science, INFORMS, vol. 68(4), pages 2970-2986, April.
    14. repec:cup:judgdm:v:14:y:2019:i:4:p:381-394 is not listed on IDEAS
    15. Nagel, Rosemarie & Bühren, Christoph & Frank, Björn, 2017. "Inspired and inspiring: Hervé Moulin and the discovery of the beauty contest game," Mathematical Social Sciences, Elsevier, vol. 90(C), pages 191-207.
    16. Ryan Webb & Paul W. Glimcher & Kenway Louie, 2021. "The Normalization of Consumer Valuations: Context-Dependent Preferences from Neurobiological Constraints," Management Science, INFORMS, vol. 67(1), pages 93-125, January.
    17. S. Cerreia-Vioglio & F. Maccheroni & M. Marinacci & A. Rustichini, 2017. "Multinomial logit processes and preference discovery: inside and outside the black box," Working Papers 615, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    18. Grabiszewski, Konrad & Horenstein, Alex, 2022. "Measuring tree complexity with response times," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 98(C).
    19. Andrew Schotter & Isabel Trevino, 2021. "Is response time predictive of choice? An experimental study of threshold strategies," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 87-117, March.
    20. Xavier Gabaix, 2017. "Behavioral Inattention," NBER Working Papers 24096, National Bureau of Economic Research, Inc.
    21. David J. Cooper & Ian Krajbich & Charles N. Noussair, 2019. "Choice-Process Data in Experimental Economics," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 1-13, August.

    More about this item

    Keywords

    Drift diffusion; Neuroeconomics; Prediction; Response times;
    All these keywords.

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • D03 - Microeconomics - - General - - - Behavioral Microeconomics: Underlying Principles
    • D87 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Neuroeconomics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jeborg:v:148:y:2018:i:c:p:344-375. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jebo .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.