IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v48y2023i1p103-141.html
   My bibliography  Save this article

Development of a High-Accuracy and Effective Online Calibration Method in CD-CAT Based on Gini Index

Author

Listed:
  • Qingrong Tan

    (Jiangxi Normal University
    Army Medical University)

  • Yan Cai
  • Fen Luo
  • Dongbo Tu

    (Jiangxi Normal University)

Abstract

To improve the calibration accuracy and calibration efficiency of cognitive diagnostic computerized adaptive testing (CD-CAT) for new items and, ultimately, contribute to the widespread application of CD-CAT in practice, the current article proposed a Gini-based online calibration method that can simultaneously calibrate the Q-matrix and item parameters of new items. Three simulation studies with simulated and real item banks were conducted to investigate the performance of the proposed method and compare it with the joint estimation algorithm (JEA) and the single-item estimation (SIE) methods. The results indicated that the proposed Gini-based online calibration method yielded higher calibration efficiency than those of the SIE method and outperformed the JEA method on item calibration tasks in terms of both accuracy and efficiency under most experimental conditions.

Suggested Citation

  • Qingrong Tan & Yan Cai & Fen Luo & Dongbo Tu, 2023. "Development of a High-Accuracy and Effective Online Calibration Method in CD-CAT Based on Gini Index," Journal of Educational and Behavioral Statistics, , vol. 48(1), pages 103-141, February.
    • Handle: RePEc:sae:jedbes:v:48:y:2023:i:1:p:103-141
    DOI: 10.3102/10769986221126741
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986221126741
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986221126741?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Strobl, Carolin & Boulesteix, Anne-Laure & Augustin, Thomas, 2007. "Unbiased split selection for classification trees based on the Gini Index," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 483-501, September.
    2. Hong-Yun Liu & Xiao-Feng You & Wen-Yi Wang & Shu-Liang Ding & Hua-Hua Chang, 2013. "The Development of Computerized Adaptive Testing with Cognitive Diagnosis for an English Achievement Test in China," Journal of Classification, Springer;The Classification Society, vol. 30(2), pages 152-172, July.
    3. Ping Chen & Tao Xin & Chun Wang & Hua-Hua Chang, 2012. "Online Calibration Methods for the DINA Model with Independent Attributes in CD-CAT," Psychometrika, Springer;The Psychometric Society, vol. 77(2), pages 201-222, April.
    4. Jimmy de la Torre, 2011. "The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(2), pages 179-199, April.
    5. Jimmy Torre, 2011. "Erratum to: The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(3), pages 510-510, July.
    6. Ying Cheng, 2009. "When Cognitive Diagnosis Meets Computerized Adaptive Testing: CD-CAT," Psychometrika, Springer;The Psychometric Society, vol. 74(4), pages 619-632, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chia-Yi Chiu & Yuan-Pei Chang, 2021. "Advances in CD-CAT: The General Nonparametric Item Selection Method," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 1039-1057, December.
    2. Wenyi Wang & Lihong Song & Teng Wang & Peng Gao & Jian Xiong, 2020. "A Note on the Relationship of the Shannon Entropy Procedure and the Jensen–Shannon Divergence in Cognitive Diagnostic Computerized Adaptive Testing," SAGE Open, , vol. 10(1), pages 21582440198, January.
    3. Ragip Terzi & Sedat Sen, 2019. "A Nondiagnostic Assessment for Diagnostic Purposes: Q-Matrix Validation and Item-Based Model Fit Evaluation for the TIMSS 2011 Assessment," SAGE Open, , vol. 9(1), pages 21582440198, February.
    4. Xuliang Gao & Daxun Wang & Yan Cai & Dongbo Tu, 2020. "Cognitive Diagnostic Computerized Adaptive Testing for Polytomously Scored Items," Journal of Classification, Springer;The Classification Society, vol. 37(3), pages 709-729, October.
    5. Matthew S. Johnson & Sandip Sinharay, 2020. "The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models," Journal of Educational and Behavioral Statistics, , vol. 45(1), pages 5-31, February.
    6. Chenchen Ma & Jing Ouyang & Gongjun Xu, 2023. "Learning Latent and Hierarchical Structures in Cognitive Diagnosis Models," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 175-207, March.
    7. James Joseph Balamuta & Steven Andrew Culpepper, 2022. "Exploratory Restricted Latent Class Models with Monotonicity Requirements under PÒLYA–GAMMA Data Augmentation," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 903-945, September.
    8. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference for the DINA Model," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 569-597, October.
    9. Xin Xu & Guanhua Fang & Jinxin Guo & Zhiliang Ying & Susu Zhang, 2024. "Diagnostic Classification Models for Testlets: Methods and Theory," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 851-876, September.
    10. Laine Bradshaw & Jonathan Templin, 2014. "Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions," Psychometrika, Springer;The Psychometric Society, vol. 79(3), pages 403-425, July.
    11. Peida Zhan & Wen-Chung Wang & Xiaomin Li, 2020. "A Partial Mastery, Higher-Order Latent Structural Model for Polytomous Attributes in Cognitive Diagnostic Assessments," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 328-351, July.
    12. Peida Zhan & Hong Jiao & Dandan Liao & Feiming Li, 2019. "A Longitudinal Higher-Order Diagnostic Classification Model," Journal of Educational and Behavioral Statistics, , vol. 44(3), pages 251-281, June.
    13. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference Algorithm for the Saturated Diagnostic Classification Model," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 973-995, December.
    14. Yan Li & Chao Huang & Jia Liu, 2023. "Diagnosing Primary Students’ Reading Progression: Is Cognitive Diagnostic Computerized Adaptive Testing the Way Forward?," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 842-865, December.
    15. Chen-Wei Liu & Björn Andersson & Anders Skrondal, 2020. "A Constrained Metropolis–Hastings Robbins–Monro Algorithm for Q Matrix Estimation in DINA Models," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 322-357, June.
    16. Chun Wang & Jing Lu, 2021. "Learning Attribute Hierarchies From Data: Two Exploratory Approaches," Journal of Educational and Behavioral Statistics, , vol. 46(1), pages 58-84, February.
    17. Hans Friedrich Köhn & Chia-Yi Chiu, 2021. "A Unified Theory of the Completeness of Q-Matrices for the DINA Model," Journal of Classification, Springer;The Classification Society, vol. 38(3), pages 500-518, October.
    18. Xuliang Gao & Wenchao Ma & Daxun Wang & Yan Cai & Dongbo Tu, 2021. "A Class of Cognitive Diagnosis Models for Polytomous Data," Journal of Educational and Behavioral Statistics, , vol. 46(3), pages 297-322, June.
    19. Muhamad Firdaus Mohd Noh & Mohd Effendi Ewan bin Mohd Matore, 2024. "Conundrum and Considerations in Cognitive Diagnostic Assessment for Language Proficiency Evaluation," Information Management and Business Review, AMH International, vol. 16(2), pages 63-72.
    20. Kevin Carl P. Santos & Jimmy Torre & Matthias Davier, 2020. "Adjusting Person Fit Index for Skewness in Cognitive Diagnosis Modeling," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 399-420, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:48:y:2023:i:1:p:103-141. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.