IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v84y2019i2d10.1007_s11336-018-9643-8.html
   My bibliography  Save this article

Estimating the Cognitive Diagnosis $$\varvec{Q}$$ Q Matrix with Expert Knowledge: Application to the Fraction-Subtraction Dataset

Author

Listed:
  • Steven Andrew Culpepper

    (University of Illinois at Urbana-Champaign)

Abstract

Cognitive diagnosis models (CDMs) are an important psychometric framework for classifying students in terms of attribute and/or skill mastery. The $$\varvec{Q}$$ Q matrix, which specifies the required attributes for each item, is central to implementing CDMs. The general unavailability of $$\varvec{Q}$$ Q for most content areas and datasets poses a barrier to widespread applications of CDMs, and recent research accordingly developed fully exploratory methods to estimate Q. However, current methods do not always offer clear interpretations of the uncovered skills and existing exploratory methods do not use expert knowledge to estimate Q. We consider Bayesian estimation of $$\varvec{Q}$$ Q using a prior based upon expert knowledge using a fully Bayesian formulation for a general diagnostic model. The developed method can be used to validate which of the underlying attributes are predicted by experts and to identify residual attributes that remain unexplained by expert knowledge. We report Monte Carlo evidence about the accuracy of selecting active expert-predictors and present an application using Tatsuoka’s fraction-subtraction dataset.

Suggested Citation

  • Steven Andrew Culpepper, 2019. "Estimating the Cognitive Diagnosis $$\varvec{Q}$$ Q Matrix with Expert Knowledge: Application to the Fraction-Subtraction Dataset," Psychometrika, Springer;The Psychometric Society, vol. 84(2), pages 333-357, June.
    • Handle: RePEc:spr:psycho:v:84:y:2019:i:2:d:10.1007_s11336-018-9643-8
    DOI: 10.1007/s11336-018-9643-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-018-9643-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-018-9643-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Chen, Yunxiao & Liu, Jingchen & Xu, Gongjun & Ying, Zhiliang, 2015. "Statistical analysis of Q-matrix based diagnostic classification models," LSE Research Online Documents on Economics 103183, London School of Economics and Political Science, LSE Library.
    2. A. Béguin & C. Glas, 2001. "MCMC estimation and some model-fit analysis of multidimensional IRT models," Psychometrika, Springer;The Psychometric Society, vol. 66(4), pages 541-561, December.
    3. Steven Andrew Culpepper, 2016. "Revisiting the 4-Parameter Item Response Model: Bayesian Estimation and Application," Psychometrika, Springer;The Psychometric Society, vol. 81(4), pages 1142-1163, December.
    4. Gongjun Xu & Zhuoran Shang, 2018. "Identifying Latent Structures in Restricted Latent Class Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1284-1295, July.
    5. Jingchen Liu, 2017. "On the Consistency of Q-Matrix Estimation: A Commentary," Psychometrika, Springer;The Psychometric Society, vol. 82(2), pages 523-527, June.
    6. Curtis Tatsuoka, 2002. "Data analytic methods for latent partially ordered classification models," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 51(3), pages 337-350, July.
    7. David J. Spiegelhalter & Nicola G. Best & Bradley P. Carlin & Angelika Van Der Linde, 2002. "Bayesian measures of model complexity and fit," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(4), pages 583-639, October.
    8. James H. Albert, 1992. "Bayesian Estimation of Normal Ogive Item Response Curves Using Gibbs Sampling," Journal of Educational and Behavioral Statistics, , vol. 17(3), pages 251-269, September.
    9. Jimmy de la Torre, 2011. "The Generalized DINA Model Framework," Psychometrika, Springer;The Psychometric Society, vol. 76(2), pages 179-199, April.
    10. Steven Andrew Culpepper, 2015. "Bayesian Estimation of the DINA Model With Gibbs Sampling," Journal of Educational and Behavioral Statistics, , vol. 40(5), pages 454-476, October.
    11. E. Maris, 1999. "Estimating multiple classification latent class models," Psychometrika, Springer;The Psychometric Society, vol. 64(2), pages 187-212, June.
    12. Jimmy Torre & Jeffrey Douglas, 2004. "Higher-order latent trait models for cognitive diagnosis," Psychometrika, Springer;The Psychometric Society, vol. 69(3), pages 333-353, September.
    13. Chia-Yi Chiu & Jeffrey Douglas & Xiaodong Li, 2009. "Cluster Analysis for Cognitive Diagnosis: Theory and Applications," Psychometrika, Springer;The Psychometric Society, vol. 74(4), pages 633-665, December.
    14. Yunxiao Chen & Jingchen Liu & Gongjun Xu & Zhiliang Ying, 2015. "Statistical Analysis of Q -Matrix Based Diagnostic Classification Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(510), pages 850-866, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Steven Andrew Culpepper, 2023. "A Note on Weaker Conditions for Identifying Restricted Latent Class Models for Binary Responses," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 158-174, March.
    2. Chenchen Ma & Jing Ouyang & Gongjun Xu, 2023. "Learning Latent and Hierarchical Structures in Cognitive Diagnosis Models," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 175-207, March.
    3. Chen-Wei Liu & Björn Andersson & Anders Skrondal, 2020. "A Constrained Metropolis–Hastings Robbins–Monro Algorithm for Q Matrix Estimation in DINA Models," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 322-357, June.
    4. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference for the DINA Model," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 569-597, October.
    5. James Joseph Balamuta & Steven Andrew Culpepper, 2022. "Exploratory Restricted Latent Class Models with Monotonicity Requirements under PÒLYA–GAMMA Data Augmentation," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 903-945, September.
    6. Chengcheng Li & Chenchen Ma & Gongjun Xu, 2022. "Learning Large Q-Matrix by Restricted Boltzmann Machines," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 1010-1041, September.
    7. Kazuhiro Yamaguchi & Jonathan Templin, 2022. "A Gibbs Sampling Algorithm with Monotonicity Constraints for Diagnostic Classification Models," Journal of Classification, Springer;The Classification Society, vol. 39(1), pages 24-54, March.
    8. Motonori Oka & Kensuke Okada, 2023. "Scalable Bayesian Approach for the Dina Q-Matrix Estimation Combining Stochastic Optimization and Variational Inference," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 302-331, March.
    9. Chenchen Ma & Jimmy Torre & Gongjun Xu, 2023. "Bridging Parametric and Nonparametric Methods in Cognitive Diagnosis," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 51-75, March.
    10. Yinyin Chen & Steven Culpepper & Feng Liang, 2020. "A Sparse Latent Class Model for Cognitive Diagnosis," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 121-153, March.
    11. Jimmy de la Torre & Xue-Lan Qiu & Kevin Carl Santos, 2022. "An Empirical Q-Matrix Validation Method for the Polytomous G-DINA Model," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 693-724, June.
    12. Yinghan Chen & Ying Liu & Steven Andrew Culpepper & Yuguo Chen, 2021. "Inferring the Number of Attributes for the Exploratory DINA Model," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 30-64, March.
    13. Kazuhiro Yamaguchi & Jonathan Templin, 2022. "Direct Estimation of Diagnostic Classification Model Attribute Mastery Profiles via a Collapsed Gibbs Sampling Algorithm," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1390-1421, December.
    14. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference Algorithm for the Saturated Diagnostic Classification Model," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 973-995, December.
    15. Steven Andrew Culpepper, 2019. "An Exploratory Diagnostic Model for Ordinal Responses with Binary Attributes: Identifiability and Estimation," Psychometrika, Springer;The Psychometric Society, vol. 84(4), pages 921-940, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Steven Andrew Culpepper, 2019. "An Exploratory Diagnostic Model for Ordinal Responses with Binary Attributes: Identifiability and Estimation," Psychometrika, Springer;The Psychometric Society, vol. 84(4), pages 921-940, December.
    2. Peida Zhan & Wen-Chung Wang & Xiaomin Li, 2020. "A Partial Mastery, Higher-Order Latent Structural Model for Polytomous Attributes in Cognitive Diagnostic Assessments," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 328-351, July.
    3. Yinghan Chen & Steven Andrew Culpepper & Yuguo Chen & Jeffrey Douglas, 2018. "Bayesian Estimation of the DINA Q matrix," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 89-108, March.
    4. Yuqi Gu & Jingchen Liu & Gongjun Xu & Zhiliang Ying, 2018. "Hypothesis Testing of the Q-matrix," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 515-537, September.
    5. Yinyin Chen & Steven Culpepper & Feng Liang, 2020. "A Sparse Latent Class Model for Cognitive Diagnosis," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 121-153, March.
    6. Steven Andrew Culpepper & Yinghan Chen, 2019. "Development and Application of an Exploratory Reduced Reparameterized Unified Model," Journal of Educational and Behavioral Statistics, , vol. 44(1), pages 3-24, February.
    7. Motonori Oka & Kensuke Okada, 2023. "Scalable Bayesian Approach for the Dina Q-Matrix Estimation Combining Stochastic Optimization and Variational Inference," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 302-331, March.
    8. James Joseph Balamuta & Steven Andrew Culpepper, 2022. "Exploratory Restricted Latent Class Models with Monotonicity Requirements under PÒLYA–GAMMA Data Augmentation," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 903-945, September.
    9. Yinghan Chen & Ying Liu & Steven Andrew Culpepper & Yuguo Chen, 2021. "Inferring the Number of Attributes for the Exploratory DINA Model," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 30-64, March.
    10. Juntao Wang & Yuan Li, 2023. "DINA Model with Entropy Penalization," Mathematics, MDPI, vol. 11(18), pages 1-16, September.
    11. Steven Andrew Culpepper, 2023. "A Note on Weaker Conditions for Identifying Restricted Latent Class Models for Binary Responses," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 158-174, March.
    12. Chen-Wei Liu & Björn Andersson & Anders Skrondal, 2020. "A Constrained Metropolis–Hastings Robbins–Monro Algorithm for Q Matrix Estimation in DINA Models," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 322-357, June.
    13. Chun Wang & Jing Lu, 2021. "Learning Attribute Hierarchies From Data: Two Exploratory Approaches," Journal of Educational and Behavioral Statistics, , vol. 46(1), pages 58-84, February.
    14. Hans Friedrich Köhn & Chia-Yi Chiu, 2021. "A Unified Theory of the Completeness of Q-Matrices for the DINA Model," Journal of Classification, Springer;The Classification Society, vol. 38(3), pages 500-518, October.
    15. Jimmy de la Torre & Xue-Lan Qiu & Kevin Carl Santos, 2022. "An Empirical Q-Matrix Validation Method for the Polytomous G-DINA Model," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 693-724, June.
    16. Yinghan Chen & Steven Andrew Culpepper & Yuguo Chen, 2023. "Bayesian Inference for an Unknown Number of Attributes in Restricted Latent Class Models," Psychometrika, Springer;The Psychometric Society, vol. 88(2), pages 613-635, June.
    17. Guanhua Fang & Jingchen Liu & Zhiliang Ying, 2019. "On the Identifiability of Diagnostic Classification Models," Psychometrika, Springer;The Psychometric Society, vol. 84(1), pages 19-40, March.
    18. Xin Xu & Guanhua Fang & Jinxin Guo & Zhiliang Ying & Susu Zhang, 2024. "Diagnostic Classification Models for Testlets: Methods and Theory," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 851-876, September.
    19. Chenchen Ma & Jimmy Torre & Gongjun Xu, 2023. "Bridging Parametric and Nonparametric Methods in Cognitive Diagnosis," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 51-75, March.
    20. Yinghan Chen & Shiyu Wang, 2023. "Bayesian Estimation of Attribute Hierarchy for Cognitive Diagnosis Models," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 810-841, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:84:y:2019:i:2:d:10.1007_s11336-018-9643-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.