IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v88y2023i1d10.1007_s11336-022-09880-8.html
   My bibliography  Save this article

Accurate Assessment via Process Data

Author

Listed:
  • Susu Zhang

    (University of Illinois at Urbana-Champaign)

  • Zhi Wang

    (Citadel Securities)

  • Jitong Qi

    (Columbia University)

  • Jingchen Liu

    (Columbia University)

  • Zhiliang Ying

    (Columbia University)

Abstract

Accurate assessment of a student’s ability is the key task of a test. Assessments based on final responses are the standard. As the infrastructure advances, substantially more information is observed. One of such instances is the process data that is collected by computer-based interactive items and contain a student’s detailed interactive processes. In this paper, we show both theoretically and with simulated and empirical data that appropriately including such information in the assessment will substantially improve relevant assessment precision.

Suggested Citation

  • Susu Zhang & Zhi Wang & Jitong Qi & Jingchen Liu & Zhiliang Ying, 2023. "Accurate Assessment via Process Data," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 76-97, March.
    • Handle: RePEc:spr:psycho:v:88:y:2023:i:1:d:10.1007_s11336-022-09880-8
    DOI: 10.1007/s11336-022-09880-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-022-09880-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-022-09880-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Michelle M. LaMar, 2018. "Markov Decision Process Measurement Model," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 67-88, March.
    2. Wim van der Linden, 2007. "A Hierarchical Framework for Modeling Speed and Accuracy on Test Items," Psychometrika, Springer;The Psychometric Society, vol. 72(3), pages 287-308, September.
    3. Xueying Tang & Zhi Wang & Qiwei He & Jingchen Liu & Zhiliang Ying, 2020. "Latent Feature Extraction for Process Data via Multidimensional Scaling," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 378-397, June.
    4. Norman Rose & Matthias Davier & Benjamin Nagengast, 2017. "Modeling Omitted and Not-Reached Items in IRT Models," Psychometrika, Springer;The Psychometric Society, vol. 82(3), pages 795-819, September.
    5. Jwa Kim & W. Nicewander, 1993. "Ability estimation for conventional tests," Psychometrika, Springer;The Psychometric Society, vol. 58(4), pages 587-599, December.
    6. Xueying Tang & Susu Zhang & Zhi Wang & Jingchen Liu & Zhiliang Ying, 2021. "ProcData: An R Package for Process Data Analysis," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 1058-1083, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jinxin Guo & Xin Xu & Zhiliang Ying & Susu Zhang, 2022. "Modeling Not-Reached Items in Timed Tests: A Response Time Censoring Approach," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 835-867, September.
    2. Chun Wang, 2015. "On Latent Trait Estimation in Multidimensional Compensatory Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 428-449, June.
    3. Peida Zhan & Xin Qiao, 2022. "DIAGNOSTIC Classification Analysis of Problem-Solving Competence using Process Data: An Item Expansion Method," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1529-1547, December.
    4. Pujue Wang & Hongyun Liu, 2024. "Polytomous Effectiveness Indicators in Complex Problem-Solving Tasks and Their Applications in Developing Measurement Model," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 877-902, September.
    5. Esther Ulitzsch & Qiwei He & Vincent Ulitzsch & Hendrik Molter & André Nichterlein & Rolf Niedermeier & Steffi Pohl, 2021. "Combining Clickstream Analyses and Graph-Modeled Data Clustering for Identifying Common Response Processes," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 190-214, March.
    6. Yunxiao Chen, 2020. "A Continuous-Time Dynamic Choice Measurement Model for Problem-Solving Process Data," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 1052-1075, December.
    7. Steven Andrew Culpepper & James Joseph Balamuta, 2017. "A Hierarchical Model for Accuracy and Choice on Standardized Tests," Psychometrika, Springer;The Psychometric Society, vol. 82(3), pages 820-845, September.
    8. Inhan Kang & Paul Boeck & Roger Ratcliff, 2022. "Modeling Conditional Dependence of Response Accuracy and Response Time with the Diffusion Item Response Theory Model," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 725-748, June.
    9. Федерякин Д. А., 2020. "Межстрановая Сопоставимость Результатов Тестирования В Международных Сравнительных Исследованиях Высшего Образования," Вопросы образования // Educational Studies Moscow, National Research University Higher School of Economics, issue 2, pages 37-59.
    10. Sandip Sinharay, 2022. "Reporting Proficiency Levels for Examinees With Incomplete Data," Journal of Educational and Behavioral Statistics, , vol. 47(3), pages 263-296, June.
    11. Edison M. Choe & Jinming Zhang & Hua-Hua Chang, 2018. "Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 650-673, September.
    12. Onur Demirkaya & Ummugul Bezirhan & Jinming Zhang, 2023. "Detecting Item Preknowledge Using Revisits With Speed and Accuracy," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 521-542, August.
    13. Jochen Ranger, 2013. "A Note on the Hierarchical Model for Responses and Response Times in Tests of van der Linden (2007)," Psychometrika, Springer;The Psychometric Society, vol. 78(3), pages 538-544, July.
    14. Simon Grund & Oliver Lüdtke & Alexander Robitzsch, 2021. "On the Treatment of Missing Data in Background Questionnaires in Educational Large-Scale Assessments: An Evaluation of Different Procedures," Journal of Educational and Behavioral Statistics, , vol. 46(4), pages 430-465, August.
    15. Minjeong Jeon & Paul Boeck & Jevan Luo & Xiangrui Li & Zhong-Lin Lu, 2021. "Modeling Within-Item Dependencies in Parallel Data on Test Responses and Brain Activation," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 239-271, March.
    16. Robitzsch, Alexander, 2020. "About Still Nonignorable Consequences of (Partially) Ignoring Missing Item Responses in Large-scale Assessment," OSF Preprints hmy45, Center for Open Science.
    17. Hongyue Zhu & Hong Jiao & Wei Gao & Xiangbin Meng, 2023. "Bayesian Change-Point Analysis Approach to Detecting Aberrant Test-Taking Behavior Using Response Times," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 490-520, August.
    18. Jeffrey Rouder & Jordan Province & Richard Morey & Pablo Gomez & Andrew Heathcote, 2015. "The Lognormal Race: A Cognitive-Process Model of Choice and Latency with Desirable Psychometric Properties," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 491-513, June.
    19. Steffi Pohl & Esther Ulitzsch & Matthias Davier, 2019. "Using Response Times to Model Not-Reached Items due to Time Limits," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 892-920, September.
    20. Seonghoon Kim, 2012. "A Note on the Reliability Coefficients for Item Response Model-Based Ability Estimates," Psychometrika, Springer;The Psychometric Society, vol. 77(1), pages 153-162, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:88:y:2023:i:1:d:10.1007_s11336-022-09880-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.