IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v84y2019i4d10.1007_s11336-019-09682-5.html
   My bibliography  Save this article

Modeling Differences Between Response Times of Correct and Incorrect Responses

Author

Listed:
  • Maria Bolsinova

    (ACTNext)

  • Jesper Tijmstra

    (Tilburg University)

Abstract

While standard joint models for response time and accuracy commonly assume the relationship between response time and accuracy to be fully explained by the latent variables of the model, this assumption of conditional independence is often violated in practice. If such violations are present, taking these residual dependencies between response time and accuracy into account may both improve the fit of the model to the data and improve our understanding of the response processes that led to the observed responses. In this paper, we propose a framework for the joint modeling of response time and accuracy data that allows for differences in the processes leading to correct and incorrect responses. Extensions of the standard hierarchical model (van der Linden in Psychometrika 72:287–308, 2007. https://doi.org/10.1007/s11336-006-1478-z) are considered that allow some or all item parameters in the measurement model of speed to differ depending on whether a correct or an incorrect response was obtained. The framework also allows one to consider models that include two speed latent variables, which explain the patterns observed in the responses times of correct and of incorrect responses, respectively. Model selection procedures are proposed and evaluated based on a simulation study, and a simulation study investigating parameter recovery is presented. An application of the modeling framework to empirical data from international large-scale assessment is considered to illustrate the relevance of modeling possible differences between the processes leading to correct and incorrect responses.

Suggested Citation

  • Maria Bolsinova & Jesper Tijmstra, 2019. "Modeling Differences Between Response Times of Correct and Incorrect Responses," Psychometrika, Springer;The Psychometric Society, vol. 84(4), pages 1018-1046, December.
  • Handle: RePEc:spr:psycho:v:84:y:2019:i:4:d:10.1007_s11336-019-09682-5
    DOI: 10.1007/s11336-019-09682-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-019-09682-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11336-019-09682-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wim J. van der Linden, 2006. "A Lognormal Model for Response Times on Test Items," Journal of Educational and Behavioral Statistics, , vol. 31(2), pages 181-204, June.
    2. Jeffrey Rouder & Dongchu Sun & Paul Speckman & Jun Lu & Duo Zhou, 2003. "A hierarchical bayesian statistical framework for response time distributions," Psychometrika, Springer;The Psychometric Society, vol. 68(4), pages 589-606, December.
    3. Maria Bolsinova & Jesper Tijmstra, 2016. "Posterior Predictive Checks for Conditional Independence Between Response Time and Accuracy," Journal of Educational and Behavioral Statistics, , vol. 41(2), pages 123-145, April.
    4. Wim Linden & Cees Glas, 2010. "Statistical Tests of Conditional Independence Between Responses and/or Response Times on Test Items," Psychometrika, Springer;The Psychometric Society, vol. 75(1), pages 120-139, March.
    5. Wim van der Linden, 2007. "A Hierarchical Framework for Modeling Speed and Accuracy on Test Items," Psychometrika, Springer;The Psychometric Society, vol. 72(3), pages 287-308, September.
    6. Chun Wang & Zhewen Fan & Hua-Hua Chang & Jeffrey A. Douglas, 2013. "A Semiparametric Model for Jointly Analyzing Response Times and Accuracy in Computerized Testing," Journal of Educational and Behavioral Statistics, , vol. 38(4), pages 381-417, August.
    7. Maria Bolsinova & Paul Boeck & Jesper Tijmstra, 2017. "Modelling Conditional Dependence Between Response Time and Accuracy," Psychometrika, Springer;The Psychometric Society, vol. 82(4), pages 1126-1148, December.
    8. R. Klein Entink & J.-P. Fox & W. Linden, 2009. "A Multivariate Multilevel Approach to the Modeling of Accuracy and Speed of Test Takers," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 21-48, March.
    9. Frank Goldhammer & Thomas Martens & Gabriela Christoph & Oliver Lüdtke, 2016. "Test-taking engagement in PIAAC," OECD Education Working Papers 133, OECD Publishing.
    10. James H. Albert, 1992. "Bayesian Estimation of Normal Ogive Item Response Curves Using Gibbs Sampling," Journal of Educational and Behavioral Statistics, , vol. 17(3), pages 251-269, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Maria Bolsinova & Paul Boeck & Jesper Tijmstra, 2017. "Modelling Conditional Dependence Between Response Time and Accuracy," Psychometrika, Springer;The Psychometric Society, vol. 82(4), pages 1126-1148, December.
    2. Chun Wang & Gongjun Xu & Zhuoran Shang, 2018. "A Two-Stage Approach to Differentiating Normal and Aberrant Behavior in Computer Based Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 223-254, March.
    3. Chen, Haiqin & De Boeck, Paul & Grady, Matthew & Yang, Chien-Lin & Waldschmidt, David, 2018. "Curvilinear dependency of response accuracy on response time in cognitive tests," Intelligence, Elsevier, vol. 69(C), pages 16-23.
    4. Minjeong Jeon & Paul Boeck & Jevan Luo & Xiangrui Li & Zhong-Lin Lu, 2021. "Modeling Within-Item Dependencies in Parallel Data on Test Responses and Brain Activation," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 239-271, March.
    5. Kang, Inhan & De Boeck, Paul & Partchev, Ivailo, 2022. "A randomness perspective on intelligence processes," Intelligence, Elsevier, vol. 91(C).
    6. Inhan Kang & Paul Boeck & Roger Ratcliff, 2022. "Modeling Conditional Dependence of Response Accuracy and Response Time with the Diffusion Item Response Theory Model," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 725-748, June.
    7. Steffi Pohl & Esther Ulitzsch & Matthias Davier, 2019. "Using Response Times to Model Not-Reached Items due to Time Limits," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 892-920, September.
    8. Dylan Molenaar & Paul Boeck, 2018. "Response Mixture Modeling: Accounting for Heterogeneity in Item Characteristics across Response Times," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 279-297, June.
    9. Inhan Kang & Minjeong Jeon & Ivailo Partchev, 2023. "A Latent Space Diffusion Item Response Theory Model to Explore Conditional Dependence between Responses and Response Times," Psychometrika, Springer;The Psychometric Society, vol. 88(3), pages 830-864, September.
    10. Jinxin Guo & Xin Xu & Zhiliang Ying & Susu Zhang, 2022. "Modeling Not-Reached Items in Timed Tests: A Response Time Censoring Approach," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 835-867, September.
    11. Hua-Hua Chang, 2015. "Psychometrics Behind Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 1-20, March.
    12. M. Marsman & H. Sigurdardóttir & M. Bolsinova & G. Maris, 2019. "Characterizing the Manifest Probability Distributions of Three Latent Trait Models for Accuracy and Response Time," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 870-891, September.
    13. Steven Andrew Culpepper & James Joseph Balamuta, 2017. "A Hierarchical Model for Accuracy and Choice on Standardized Tests," Psychometrika, Springer;The Psychometric Society, vol. 82(3), pages 820-845, September.
    14. Edison M. Choe & Jinming Zhang & Hua-Hua Chang, 2018. "Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 650-673, September.
    15. Sun-Joo Cho & Sarah Brown-Schmidt & Paul De Boeck & Matthew Naveiras & Si On Yoon & Aaron Benjamin, 2023. "Incorporating Functional Response Time Effects into a Signal Detection Theory Model," Psychometrika, Springer;The Psychometric Society, vol. 88(3), pages 1056-1086, September.
    16. Sandip Sinharay & Peter W. van Rijn, 2020. "Assessing Fit of the Lognormal Model for Response Times," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 534-568, October.
    17. Федерякин Д. А., 2020. "Межстрановая Сопоставимость Результатов Тестирования В Международных Сравнительных Исследованиях Высшего Образования," Вопросы образования // Educational Studies Moscow, National Research University Higher School of Economics, issue 2, pages 37-59.
    18. Inhan Kang & Dylan Molenaar & Roger Ratcliff, 2023. "A Modeling Framework to Examine Psychological Processes Underlying Ordinal Responses and Response Times of Psychometric Data," Psychometrika, Springer;The Psychometric Society, vol. 88(3), pages 940-974, September.
    19. Jeffrey Rouder & Jordan Province & Richard Morey & Pablo Gomez & Andrew Heathcote, 2015. "The Lognormal Race: A Cognitive-Process Model of Choice and Latency with Desirable Psychometric Properties," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 491-513, June.
    20. Shaw, Amy & Elizondo, Fabian & Wadlington, Patrick L., 2020. "Reasoning, fast and slow: How noncognitive factors may alter the ability-speed relationship," Intelligence, Elsevier, vol. 83(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:84:y:2019:i:4:d:10.1007_s11336-019-09682-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.