IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v39y2014i6p426-451.html
   My bibliography  Save this article

Testing for Aberrant Behavior in Response Time Modeling

Author

Listed:
  • Sukaesi Marianti
  • Jean-Paul Fox
  • Marianna Avetisyan
  • Bernard P. Veldkamp

    (University of Twente)

  • Jesper Tijmstra

Abstract

Many standardized tests are now administered via computer rather than paper-and-pencil format. In a computer-based testing environment, it is possible to record not only the test taker’s response to each question (item) but also the amount of time spent by the test taker in considering and answering each item. Response times (RTs) provide information not only about the test taker’s ability and response behavior but also about item and test characteristics. This study focuses on the use of RTs to detect aberrant test-taker responses. An example of such aberrance is a correct answer with a short RT on a difficult question. Such aberrance may be displayed when a test taker or test takers have preknowledge of the items. Another example is rapid guessing, wherein the test taker displays unusually short RTs for a series of items. When rapid guessing occurs at the end of a timed test, it often indicates that the test taker has run out of time before completing the test. In this study, Bayesian tests of significance for detecting various types of aberrant RT patterns are proposed and evaluated. In a simulation study, the tests were successful in identifying aberrant response patterns. A real data example is given to illustrate the use of the proposed person-fit tests for RTs.

Suggested Citation

  • Sukaesi Marianti & Jean-Paul Fox & Marianna Avetisyan & Bernard P. Veldkamp & Jesper Tijmstra, 2014. "Testing for Aberrant Behavior in Response Time Modeling," Journal of Educational and Behavioral Statistics, , vol. 39(6), pages 426-451, December.
    • Handle: RePEc:sae:jedbes:v:39:y:2014:i:6:p:426-451
    DOI: 10.3102/1076998614559412
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998614559412
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998614559412?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Tom Snijders, 2001. "Asymptotic null distribution of person fit statistics with estimated person parameter," Psychometrika, Springer;The Psychometric Society, vol. 66(3), pages 331-342, September.
    2. Eric Maris, 1993. "Additive and multiplicative models for gamma distributed random variables, and their application as psychometric models for response times," Psychometrika, Springer;The Psychometric Society, vol. 58(3), pages 445-469, September.
    3. R. Klein Entink & J.-P. Fox & W. Linden, 2009. "A Multivariate Multilevel Approach to the Modeling of Accuracy and Speed of Test Takers," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 21-48, March.
    4. Fox, Jean-Paul & Entink, Rinke Klein & van der Linden, Wilm, 2007. "Modeling of Responses and Response Times with the Package cirt," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 20(i07).
    5. Ivo Molenaar & Herbert Hoijtink, 1990. "The many null distributions of person fit indices," Psychometrika, Springer;The Psychometric Society, vol. 55(1), pages 75-106, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Edison M. Choe & Jinming Zhang & Hua-Hua Chang, 2018. "Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 650-673, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. C. Glas & Anna Dagohoy, 2007. "A Person Fit Test For Irt Models For Polytomous Items," Psychometrika, Springer;The Psychometric Society, vol. 72(2), pages 159-180, June.
    2. Shaw, Amy & Elizondo, Fabian & Wadlington, Patrick L., 2020. "Reasoning, fast and slow: How noncognitive factors may alter the ability-speed relationship," Intelligence, Elsevier, vol. 83(C).
    3. Kevin Carl P. Santos & Jimmy Torre & Matthias Davier, 2020. "Adjusting Person Fit Index for Skewness in Cognitive Diagnosis Modeling," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 399-420, July.
    4. David Magis & Gilles Raîche & Sébastien Béland, 2012. "A Didactic Presentation of Snijders’s lz* Index of Person Fit With Emphasis on Response Model Selection and Ability Estimation," Journal of Educational and Behavioral Statistics, , vol. 37(1), pages 57-81, February.
    5. Sandip Sinharay, 2016. "Asymptotically Correct Standardization of Person-Fit Statistics Beyond Dichotomous Items," Psychometrika, Springer;The Psychometric Society, vol. 81(4), pages 992-1013, December.
    6. Maria Bolsinova & Jesper Tijmstra, 2016. "Posterior Predictive Checks for Conditional Independence Between Response Time and Accuracy," Journal of Educational and Behavioral Statistics, , vol. 41(2), pages 123-145, April.
    7. Rob R. Meijer & Jorge N. Tendeiro, 2012. "The Use of the lz and lz* Person-Fit Statistics and Problems Derived From Model Misspecification," Journal of Educational and Behavioral Statistics, , vol. 37(6), pages 758-766, December.
    8. Sandip Sinharay, 2015. "Assessment of Person Fit for Mixed-Format Tests," Journal of Educational and Behavioral Statistics, , vol. 40(4), pages 343-365, August.
    9. Maxwell Hong & Lizhen Lin & Ying Cheng, 2021. "Asymptotically Corrected Person Fit Statistics for Multidimensional Constructs with Simple Structure and Mixed Item Types," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 464-488, June.
    10. Herbert Hoijtink & Meinte Vollema, 2003. "Contemporary Extensions of the Rasch Model," Quality & Quantity: International Journal of Methodology, Springer, vol. 37(3), pages 263-276, August.
    11. Xiang Liu & James Yang & Hui Soo Chae & Gary Natriello, 2020. "Power Divergence Family of Statistics for Person Parameters in IRT Models," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 502-525, June.
    12. Sun-Joo Cho & Paul Boeck & Susan Embretson & Sophia Rabe-Hesketh, 2014. "Additive Multilevel Item Structure Models with Random Residuals: Item Modeling for Explanation and Item Generation," Psychometrika, Springer;The Psychometric Society, vol. 79(1), pages 84-104, January.
    13. M. Jansen & C. Glas, 2005. "Checking the Assumptions of Rasch's Model for Speed Tests," Psychometrika, Springer;The Psychometric Society, vol. 70(4), pages 671-684, December.
    14. Gerard Breukelen, 1997. "Separability of item and person parameters in response time models," Psychometrika, Springer;The Psychometric Society, vol. 62(4), pages 525-544, December.
    15. Steffi Pohl & Esther Ulitzsch & Matthias Davier, 2019. "Using Response Times to Model Not-Reached Items due to Time Limits," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 892-920, September.
    16. repec:jss:jstsof:20:i01 is not listed on IDEAS
    17. Must, Olev & Must, Aasa, 2018. "Speed and the Flynn Effect," Intelligence, Elsevier, vol. 68(C), pages 37-47.
    18. Martin Biehler & Heinz Holling & Philipp Doebler, 2015. "Saddlepoint Approximations of the Distribution of the Person Parameter in the Two Parameter Logistic Model," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 665-688, September.
    19. de Leeuw, Jan & Mair, Patrick, 2007. "An Introduction to the Special Volume on "Psychometrics in R"," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 20(i01).
    20. repec:jss:jstsof:36:c01 is not listed on IDEAS
    21. Dylan Molenaar & Paul Boeck, 2018. "Response Mixture Modeling: Accounting for Heterogeneity in Item Characteristics across Response Times," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 279-297, June.
    22. Can Shao & Jun Li & Ying Cheng, 2016. "Detection of Test Speededness Using Change-Point Analysis," Psychometrika, Springer;The Psychometric Society, vol. 81(4), pages 1118-1141, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:39:y:2014:i:6:p:426-451. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.