IDEAS home Printed from https://ideas.repec.org/p/chu/wpaper/18-16.html
   My bibliography  Save this paper

Using Response Times to Measure Ability on a Cognitive Task

Author

Listed:
  • Aleksandr Alekseev

    (Economic Science Institute, Chapman University)

Abstract

I show how using response times as a proxy for effort coupled with an explicit process-based model can address a long-standing issue of how to separate the effect of cognitive ability on performance from the effect of motivation. My method is based on a dynamic stochastic model of optimal effort choice in which ability and motivation are the structural parameters. I show how to estimate these parameters from the data on outcomes and response times in a cognitive task. In a laboratory experiment, I find that performance on a Digit-Symbol test is a noisy and biased measure of cognitive ability. Ranking subjects by their performance leads to an incorrect ranking by their ability in a substantial number of cases. These results suggest that interpreting performance on a cognitive task as ability may be misleading.

Suggested Citation

  • Aleksandr Alekseev, 2018. "Using Response Times to Measure Ability on a Cognitive Task," Working Papers 18-16, Chapman University, Economic Science Institute.
    • Handle: RePEc:chu:wpaper:18-16
    as

    Download full text from publisher

    File URL: https://digitalcommons.chapman.edu/esi_working_papers/254/
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde, 2010. "Are Risk Aversion and Impatience Related to Cognitive Ability?," American Economic Review, American Economic Association, vol. 100(3), pages 1238-1260, June.
    2. James Heckman & Rodrigo Pinto & Peter Savelyev, 2013. "Understanding the Mechanisms through Which an Influential Early Childhood Program Boosted Adult Outcomes," American Economic Review, American Economic Association, vol. 103(6), pages 2052-2086, October.
    3. Leonidas Spiliopoulos & Andreas Ortmann, 2018. "The BCD of response time analysis in experimental economics," Experimental Economics, Springer;Economic Science Association, vol. 21(2), pages 383-433, June.
    4. Clithero, John A., 2018. "Improving out-of-sample predictions using response times and a model of the decision process," Journal of Economic Behavior & Organization, Elsevier, vol. 148(C), pages 344-375.
    5. Sumit Agarwal & Bhashkar Mazumder, 2013. "Cognitive Abilities and Household Financial Decision Making," American Economic Journal: Applied Economics, American Economic Association, vol. 5(1), pages 193-207, January.
    6. Lex Borghans & Angela Lee Duckworth & James J. Heckman & Bas ter Weel, 2008. "The Economics and Psychology of Personality Traits," Journal of Human Resources, University of Wisconsin Press, vol. 43(4).
    7. Benndorf, Volker & Rau, Holger A. & Sölch, Christian, 2018. "Minimizing learning behavior in repeated real-effort tasks," University of Göttingen Working Papers in Economics 343, University of Goettingen, Department of Economics.
    8. Elie Ofek & Muhamet Yildiz & Ernan Haruvy, 2007. "The Impact of Prior Decisions on Subsequent Valuations in a Costly Contemplation Model," Management Science, INFORMS, vol. 53(8), pages 1217-1233, August.
    9. Wilcox, Nathaniel T, 1993. "Lottery Choice: Incentives, Complexity and Decision Time," Economic Journal, Royal Economic Society, vol. 103(421), pages 1397-1417, November.
    10. David Gill & Victoria Prowse, 2016. "Cognitive Ability, Character Skills, and Learning to Play Equilibrium: A Level-k Analysis," Journal of Political Economy, University of Chicago Press, vol. 124(6), pages 1619-1676.
    11. James J. Heckman & Jora Stixrud & Sergio Urzua, 2006. "The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 411-482, July.
    12. Carmit Segal, 2012. "Working When No One Is Watching: Motivation, Test Scores, and Economic Success," Management Science, INFORMS, vol. 58(8), pages 1438-1457, August.
    13. Murnane, Richard J & Willett, John B & Levy, Frank, 1995. "The Growing Importance of Cognitive Skills in Wage Determination," The Review of Economics and Statistics, MIT Press, vol. 77(2), pages 251-266, May.
    14. Michael Woodford, 2014. "Stochastic Choice: An Optimizing Neuroeconomic Model," American Economic Review, American Economic Association, vol. 104(5), pages 495-500, May.
    15. Ryan Webb, 2019. "The (Neural) Dynamics of Stochastic Choice," Management Science, INFORMS, vol. 65(1), pages 230-255, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. David J. Cooper & Ian Krajbich & Charles N. Noussair, 2019. "Choice-Process Data in Experimental Economics," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 1-13, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carlos Alós-Ferrer & Ernst Fehr & Nick Netzer, 2021. "Time Will Tell: Recovering Preferences When Choices Are Noisy," Journal of Political Economy, University of Chicago Press, vol. 129(6), pages 1828-1877.
    2. James J. Heckman & Tomáš Jagelka & Timothy D. Kautz, 2019. "Some Contributions of Economics to the Study of Personality," NBER Working Papers 26459, National Bureau of Economic Research, Inc.
    3. Proto, Eugenio & Rustichini, Aldo & Sofianos, Andis, 2014. "Higher Intelligence Groups Have Higher Cooperation Rates in the Repeated Prisoner's Dilemma," IZA Discussion Papers 8499, Institute of Labor Economics (IZA).
    4. Heckman, James J., 2011. "Integrating Personality Psychology into Economics," IZA Discussion Papers 5950, Institute of Labor Economics (IZA).
    5. Kautz, Tim & Heckman, James J. & Diris, Ron & ter Weel, Bas & Borghans, Lex, 2014. "Fostering and Measuring Skills: Improving Cognitive and Non-Cognitive Skills to Promote Lifetime Success," IZA Discussion Papers 8696, Institute of Labor Economics (IZA).
    6. María José Roa & Ignacio Garrón & Jonathan Barboza, 2019. "Financial Decisions and Financial Capabilities in the Andean Region," Journal of Consumer Affairs, Wiley Blackwell, vol. 53(2), pages 296-323, June.
    7. Gong, Xiaodong & Zhu, Rong, 2019. "Cognitive abilities, non-cognitive skills, and gambling behaviors," Journal of Economic Behavior & Organization, Elsevier, vol. 165(C), pages 51-69.
    8. Cueva, Carlos & Iturbe-Ormaetxe, Iñigo & Mata-Pérez, Esther & Ponti, Giovanni & Sartarelli, Marcello & Yu, Haihan & Zhukova, Vita, 2016. "Cognitive (ir)reflection: New experimental evidence," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 64(C), pages 81-93.
    9. Carlos Alós-Ferrer & Johannes Buckenmaier, 2021. "Cognitive sophistication and deliberation times," Experimental Economics, Springer;Economic Science Association, vol. 24(2), pages 558-592, June.
    10. Peijnenburg, Kim & Parise, Gianpaolo, 2017. "Understanding the Determinants of Financial Outcomes and Choices: The Role of Noncognitive Abilities," CEPR Discussion Papers 11900, C.E.P.R. Discussion Papers.
    11. Flavio Cunha & James J. Heckman, 2009. "The Economics and Psychology of Inequality and Human DEvelopment," Journal of the European Economic Association, MIT Press, vol. 7(2-3), pages 320-364, 04-05.
    12. Andrew Schotter & Isabel Trevino, 2021. "Is response time predictive of choice? An experimental study of threshold strategies," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 87-117, March.
    13. Balsa, Ana & Gandelman, Néstor & Roldán, Flavia, 2015. "Peer Effects in the Development of Capabilities in Adolescence," Research Department working papers 820, CAF Development Bank Of Latinamerica.
    14. Guber, Raphael, 2019. "Making it right? Social norms, handwriting and human capital," Labour Economics, Elsevier, vol. 56(C), pages 44-57.
    15. Maria Cubel & Ana Nuevo‐Chiquero & Santiago Sanchez‐Pages & Marian Vidal‐Fernandez, 2016. "Do Personality Traits Affect Productivity? Evidence from the Laboratory," Economic Journal, Royal Economic Society, vol. 0(592), pages 654-681, May.
    16. Balart, Pau & Oosterveen, Matthijs & Webbink, Dinand, 2018. "Test scores, noncognitive skills and economic growth," Economics of Education Review, Elsevier, vol. 63(C), pages 134-153.
    17. Bastani, Spencer & Karlsson, Kristina & Kolsrud, Jonas & Waldenström, Daniel, 2024. "The Capital Advantage: Comparing Returns to Ability in the Labor and Capital Markets," Working Papers in Economics and Statistics 1/2024, Linnaeus University, School of Business and Economics, Department of Economics and Statistics.
    18. Daniel Horn & Hubert Janos Kiss, 2018. "Which preferences associate with school performance?—Lessons from an exploratory study with university students," PLOS ONE, Public Library of Science, vol. 13(2), pages 1-32, February.
    19. Clithero, John A., 2018. "Response times in economics: Looking through the lens of sequential sampling models," Journal of Economic Psychology, Elsevier, vol. 69(C), pages 61-86.
    20. Shelly Lundberg, 2017. "Noncognitive Skills as Human Capital," NBER Chapters, in: Education, Skills, and Technical Change: Implications for Future US GDP Growth, pages 219-243, National Bureau of Economic Research, Inc.

    More about this item

    Keywords

    cognitive ability; test scores; response times; drift-diffusion model; choice-process data;
    All these keywords.

    JEL classification:

    • C24 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Truncated and Censored Models; Switching Regression Models; Threshold Regression Models
    • C41 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Duration Analysis; Optimal Timing Strategies
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:chu:wpaper:18-16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Megan Luetje (email available below). General contact details of provider: https://edirc.repec.org/data/esichus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.