IDEAS home Printed from https://ideas.repec.org/a/spr/stmapp/v22y2013i2p243-267.html
   My bibliography  Save this article

On the use of MCMC computerized adaptive testing with empirical prior information to improve efficiency

Author

Listed:
  • Mariagiulia Matteucci
  • Bernard Veldkamp

Abstract

The paper deals with the introduction of empirical prior information in the estimation of candidate’s ability within computerized adaptive testing (CAT). CAT is generally applied to improve efficiency of test administration. In this paper, it is shown how the inclusion of background variables both in the initialization and the ability estimation is able to improve the accuracy of ability estimates. In particular, a Gibbs sampler scheme is proposed in the phases of interim and final ability estimation. By using both simulated and real data, it is proved that the method produces more accurate ability estimates, especially for short tests and when reproducing boundary abilities. This implies that operational problems of CAT related to weak measurement precision under particular conditions, can be reduced as well. In the empirical examples, the methods were applied to CAT for intelligence testing in the area of personnel selection and to educational measurement. Other promising applications would be in the medical world, where testing efficiency is of paramount importance as well. Copyright Springer-Verlag Berlin Heidelberg 2013

Suggested Citation

  • Mariagiulia Matteucci & Bernard Veldkamp, 2013. "On the use of MCMC computerized adaptive testing with empirical prior information to improve efficiency," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 22(2), pages 243-267, June.
    • Handle: RePEc:spr:stmapp:v:22:y:2013:i:2:p:243-267
    DOI: 10.1007/s10260-012-0216-1
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10260-012-0216-1
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10260-012-0216-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. A. Béguin & C. Glas, 2001. "MCMC estimation and some model-fit analysis of multidimensional IRT models," Psychometrika, Springer;The Psychometric Society, vol. 66(4), pages 541-561, December.
    2. Daniel Segall, 1996. "Multidimensional adaptive testing," Psychometrika, Springer;The Psychometric Society, vol. 61(2), pages 331-354, June.
    3. Robert Mislevy, 1986. "Bayes modal estimation in item response models," Psychometrika, Springer;The Psychometric Society, vol. 51(2), pages 177-195, June.
    4. Aeilko Zwinderman, 1991. "A generalized rasch model for manifest predictors," Psychometrika, Springer;The Psychometric Society, vol. 56(4), pages 589-600, December.
    5. Wim Linden, 1998. "Bayesian item selection criteria for adaptive testing," Psychometrika, Springer;The Psychometric Society, vol. 63(2), pages 201-216, June.
    6. Martijn G. de Jong & Jan-Benedict E. M. Steenkamp & Bernard P. Veldkamp, 2009. "A Model for the Construction of Country-Specific Yet Internationally Comparable Short-Form Marketing Scales," Marketing Science, INFORMS, vol. 28(4), pages 674-689, 07-08.
    7. Bernard Veldkamp & Wim Linden, 2002. "Multidimensional adaptive testing with constraints on test content," Psychometrika, Springer;The Psychometric Society, vol. 67(4), pages 575-588, December.
    8. Jean-Paul Fox & Cees Glas, 2001. "Bayesian estimation of a multilevel IRT model using gibbs sampling," Psychometrika, Springer;The Psychometric Society, vol. 66(2), pages 271-288, June.
    9. Thomas Warm, 1989. "Weighted likelihood estimation of ability in item response theory," Psychometrika, Springer;The Psychometric Society, vol. 54(3), pages 427-450, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chun Wang, 2015. "On Latent Trait Estimation in Multidimensional Compensatory Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 428-449, June.
    2. Lai-Fa Hung & Wen-Chung Wang, 2012. "The Generalized Multilevel Facets Model for Longitudinal Data," Journal of Educational and Behavioral Statistics, , vol. 37(2), pages 231-255, April.
    3. Chun Wang & Hua-Hua Chang, 2011. "Item Selection in Multidimensional Computerized Adaptive Testing—Gaining Information from Different Angles," Psychometrika, Springer;The Psychometric Society, vol. 76(3), pages 363-384, July.
    4. Azevedo, Caio L.N. & Andrade, Dalton F. & Fox, Jean-Paul, 2012. "A Bayesian generalized multiple group IRT model with model-fit assessment tools," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 4399-4412.
    5. Azevedo, Caio L.N. & Bolfarine, Heleno & Andrade, Dalton F., 2011. "Bayesian inference for a skew-normal IRT model under the centred parameterization," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 353-365, January.
    6. Chun Wang & Gongjun Xu & Xue Zhang, 2019. "Correction for Item Response Theory Latent Trait Measurement Error in Linear Mixed Effects Models," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 673-700, September.
    7. Li Cai, 2010. "High-dimensional Exploratory Item Factor Analysis by A Metropolis–Hastings Robbins–Monro Algorithm," Psychometrika, Springer;The Psychometric Society, vol. 75(1), pages 33-57, March.
    8. A. Béguin & C. Glas, 2001. "MCMC estimation and some model-fit analysis of multidimensional IRT models," Psychometrika, Springer;The Psychometric Society, vol. 66(4), pages 541-561, December.
    9. Chun Wang & David J. Weiss & Zhuoran Shang, 2019. "Variable-Length Stopping Rules for Multidimensional Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 749-771, September.
    10. Mariagiulia Matteucci & Bernard Veldkamp, 2015. "The approach of power priors for ability estimation in IRT models," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 917-926, May.
    11. Michael Edwards, 2010. "A Markov Chain Monte Carlo Approach to Confirmatory Item Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 75(3), pages 474-497, September.
    12. Hanneke Geerlings & Cees Glas & Wim Linden, 2011. "Modeling Rule-Based Item Generation," Psychometrika, Springer;The Psychometric Society, vol. 76(2), pages 337-359, April.
    13. Haruhiko Ogasawara, 2013. "Asymptotic properties of the Bayes modal estimators of item parameters in item response theory," Computational Statistics, Springer, vol. 28(6), pages 2559-2583, December.
    14. Chun Wang & Hua-Hua Chang & Keith Boughton, 2011. "Kullback–Leibler Information and Its Applications in Multi-Dimensional Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 76(1), pages 13-39, January.
    15. Lihua Yao, 2012. "Multidimensional CAT Item Selection Methods for Domain Scores and Composite Scores: Theory and Applications," Psychometrika, Springer;The Psychometric Society, vol. 77(3), pages 495-523, July.
    16. Ping Chen & Chun Wang, 2016. "A New Online Calibration Method for Multidimensional Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 674-701, September.
    17. Udo Boehm & Maarten Marsman & Han L. J. Maas & Gunter Maris, 2021. "An Attention-Based Diffusion Model for Psychometric Analyses," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 938-972, December.
    18. Sandip Sinharay, 2015. "The Asymptotic Distribution of Ability Estimates," Journal of Educational and Behavioral Statistics, , vol. 40(5), pages 511-528, October.
    19. Ping Chen, 2017. "A Comparative Study of Online Item Calibration Methods in Multidimensional Computerized Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 42(5), pages 559-590, October.
    20. Yan Huo & Jimmy de la Torre & Eun-Young Mun & Su-Young Kim & Anne Ray & Yang Jiao & Helene White, 2015. "A Hierarchical Multi-Unidimensional IRT Approach for Analyzing Sparse, Multi-Group Data for Integrative Data Analysis," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 834-855, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stmapp:v:22:y:2013:i:2:p:243-267. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.