IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v37y2012i5p655-670.html
   My bibliography  Save this article

Utilizing Response Time Distributions for Item Selection in CAT

Author

Listed:
  • Zhewen Fan

    (Precision Therapeutics, Inc.)

  • Chun Wang

    (University of Illinois Urbana-Champaign)

  • Hua-Hua Chang

    (University of Illinois Urbana-Champaign)

  • Jeffrey Douglas

    (University of Illinois Urbana-Champaign)

Abstract

Traditional methods for item selection in computerized adaptive testing only focus on item information without taking into consideration the time required to answer an item. As a result, some examinees may receive a set of items that take a very long time to finish, and information is not accrued as efficiently as possible. The authors propose two item-selection criteria that utilize information from a lognormal model for response times. The first modifies the maximum information criterion to maximize information per time unit. The second is an inverse time-weighted version of a-stratification that takes advantage of the response time model, but achieves more balanced item exposure than the information-based techniques. Simulations are conducted to compare these procedures against their counterparts that ignore response times, and efficiency of estimation, time-required, and item exposure rates are assessed.

Suggested Citation

  • Zhewen Fan & Chun Wang & Hua-Hua Chang & Jeffrey Douglas, 2012. "Utilizing Response Time Distributions for Item Selection in CAT," Journal of Educational and Behavioral Statistics, , vol. 37(5), pages 655-670, October.
    • Handle: RePEc:sae:jedbes:v:37:y:2012:i:5:p:655-670
    DOI: 10.3102/1076998611422912
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998611422912
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998611422912?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Daniel Segall, 1996. "Multidimensional adaptive testing," Psychometrika, Springer;The Psychometric Society, vol. 61(2), pages 331-354, June.
    2. Daniel Segall, 2001. "General ability measurement: An application of multidimensional item response theory," Psychometrika, Springer;The Psychometric Society, vol. 66(1), pages 79-97, March.
    3. Eric Maris, 1993. "Additive and multiplicative models for gamma distributed random variables, and their application as psychometric models for response times," Psychometrika, Springer;The Psychometric Society, vol. 58(3), pages 445-469, September.
    4. Joris Mulder & Wim Linden, 2009. "Multidimensional Adaptive Testing with Optimal Design Criteria for Item Selection," Psychometrika, Springer;The Psychometric Society, vol. 74(2), pages 273-296, June.
    5. Hua-Hua Chang & William Stout, 1993. "The asymptotic posterior normality of the latent trait in an IRT model," Psychometrika, Springer;The Psychometric Society, vol. 58(1), pages 37-52, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chun Wang & Gongjun Xu & Zhuoran Shang, 2018. "A Two-Stage Approach to Differentiating Normal and Aberrant Behavior in Computer Based Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 223-254, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hua-Hua Chang, 2015. "Psychometrics Behind Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 1-20, March.
    2. Ping Chen & Chun Wang, 2016. "A New Online Calibration Method for Multidimensional Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 674-701, September.
    3. Chun Wang & Hua-Hua Chang, 2011. "Item Selection in Multidimensional Computerized Adaptive Testing—Gaining Information from Different Angles," Psychometrika, Springer;The Psychometric Society, vol. 76(3), pages 363-384, July.
    4. Chun Wang & Hua-Hua Chang & Keith Boughton, 2011. "Kullback–Leibler Information and Its Applications in Multi-Dimensional Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 76(1), pages 13-39, January.
    5. Chun Wang, 2014. "Improving Measurement Precision of Hierarchical Latent Traits Using Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 39(6), pages 452-477, December.
    6. Lihua Yao, 2012. "Multidimensional CAT Item Selection Methods for Domain Scores and Composite Scores: Theory and Applications," Psychometrika, Springer;The Psychometric Society, vol. 77(3), pages 495-523, July.
    7. Ping Chen, 2017. "A Comparative Study of Online Item Calibration Methods in Multidimensional Computerized Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 42(5), pages 559-590, October.
    8. Chun Wang & David J. Weiss & Zhuoran Shang, 2019. "Variable-Length Stopping Rules for Multidimensional Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 749-771, September.
    9. Sandip Sinharay, 2015. "The Asymptotic Distribution of Ability Estimates," Journal of Educational and Behavioral Statistics, , vol. 40(5), pages 511-528, October.
    10. Chun Wang, 2015. "On Latent Trait Estimation in Multidimensional Compensatory Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 428-449, June.
    11. Hua-Hua Chang, 1996. "The asymptotic posterior normality of the latent trait for polytomous IRT models," Psychometrika, Springer;The Psychometric Society, vol. 61(3), pages 445-463, September.
    12. Edison M. Choe & Jinming Zhang & Hua-Hua Chang, 2018. "Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 650-673, September.
    13. Daniel Segall, 2001. "General ability measurement: An application of multidimensional item response theory," Psychometrika, Springer;The Psychometric Society, vol. 66(1), pages 79-97, March.
    14. Gerard Breukelen, 1995. "Psychometric and information processing properties of selected response time models," Psychometrika, Springer;The Psychometric Society, vol. 60(1), pages 95-113, March.
    15. Sandip Sinharay & Peter W. van Rijn, 2020. "Assessing Fit of the Lognormal Model for Response Times," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 534-568, October.
    16. Anders Skrondal & Sophia Rabe‐Hesketh, 2009. "Prediction in multilevel generalized linear models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 659-687, June.
    17. Pascal Jordan, 2023. "On Reverse Shrinkage Effects and Shrinkage Overshoot," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 274-301, March.
    18. Carolyn Anderson & Hsiu-Ting Yu, 2007. "Log-Multiplicative Association Models as Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 72(1), pages 5-23, March.
    19. Ying Cheng & Cheng Liu & John Behrens, 2015. "Standard Error of Ability Estimates and the Classification Accuracy and Consistency of Binary Decisions," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 645-664, September.
    20. Jochen Ranger & Jörg-Tobias Kuhn, 2013. "Analyzing Response Times in Tests With Rank Correlation Approaches," Journal of Educational and Behavioral Statistics, , vol. 38(1), pages 61-80, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:37:y:2012:i:5:p:655-670. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.