IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v43y2018i3p259-285.html
   My bibliography  Save this article

Interval Estimation of Latent Variable Scores in Item Response Theory

Author

Listed:
  • Yang Liu
  • Ji Seung Yang

Abstract

The uncertainty arising from item parameter estimation is often not negligible and must be accounted for when calculating latent variable (LV) scores in item response theory (IRT). It is particularly so when the calibration sample size is limited and/or the calibration IRT model is complex. In the current work, we treat two-stage IRT scoring as a predictive inference problem: The target of prediction is a random variable that follows the true posterior of the LV conditional on the response pattern being scored. Various Bayesian, fiducial, and frequentist prediction intervals of LV scores, which can be obtained from a simple yet generic Monte Carlo recipe, are evaluated and contrasted via simulations based on several measures of prediction quality. An empirical data example is also presented to illustrate the use of candidate methods.

Suggested Citation

  • Yang Liu & Ji Seung Yang, 2018. "Interval Estimation of Latent Variable Scores in Item Response Theory," Journal of Educational and Behavioral Statistics, , vol. 43(3), pages 259-285, June.
    • Handle: RePEc:sae:jedbes:v:43:y:2018:i:3:p:259-285
    DOI: 10.3102/1076998617732764
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998617732764
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998617732764?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yang Liu & Jan Hannig, 2016. "Generalized Fiducial Inference for Binary Logistic Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 81(2), pages 290-324, June.
    2. Robert Tsutakawa & Jane Johnson, 1990. "The effect of uncertainty of item parameter estimation on ability estimates," Psychometrika, Springer;The Psychometric Society, vol. 55(2), pages 371-390, June.
    3. Robert Gibbons & Donald Hedeker, 1992. "Full-information item bi-factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 57(3), pages 423-436, September.
    4. Vaart,A. W. van der, 2000. "Asymptotic Statistics," Cambridge Books, Cambridge University Press, number 9780521784504, October.
    5. J. F. Lawless & Marc Fredette, 2005. "Frequentist prediction intervals and predictive distributions," Biometrika, Biometrika Trust, vol. 92(3), pages 529-542, September.
    6. Yang Liu & Jan Hannig, 2017. "Generalized Fiducial Inference for Logistic Graded Response Models," Psychometrika, Springer;The Psychometric Society, vol. 82(4), pages 1097-1125, December.
    7. Ying Cheng & Ke-Hai Yuan, 2010. "The Impact of Fallible Item Parameter Estimates on Latent Trait Recovery," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 280-291, June.
    8. Jan Hannig & Thomas C. M. Lee, 2009. "Generalized fiducial inference for wavelet regression," Biometrika, Biometrika Trust, vol. 96(4), pages 847-860.
    9. Hua-Hua Chang & William Stout, 1993. "The asymptotic posterior normality of the latent trait in an IRT model," Psychometrika, Springer;The Psychometric Society, vol. 58(1), pages 37-52, March.
    10. Jinming Zhang & Minge Xie & Xiaolan Song & Ting Lu, 2011. "Investigating the Impact of Uncertainty About Item Parameters on Ability Estimation," Psychometrika, Springer;The Psychometric Society, vol. 76(1), pages 97-118, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang Liu & Ji Seung Yang, 2018. "Bootstrap-Calibrated Interval Estimates for Latent Variable Scores in Item Response Theory," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 333-354, June.
    2. Ogasawara, Haruhiko, 2013. "Asymptotic cumulants of ability estimators using fallible item parameters," Journal of Multivariate Analysis, Elsevier, vol. 119(C), pages 144-162.
    3. C. Glas & Anna Dagohoy, 2007. "A Person Fit Test For Irt Models For Polytomous Items," Psychometrika, Springer;The Psychometric Society, vol. 72(2), pages 159-180, June.
    4. Ping Chen & Chun Wang, 2016. "A New Online Calibration Method for Multidimensional Computerized Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 674-701, September.
    5. Anders Skrondal & Sophia Rabe‐Hesketh, 2009. "Prediction in multilevel generalized linear models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 659-687, June.
    6. Ying Cheng & Cheng Liu & John Behrens, 2015. "Standard Error of Ability Estimates and the Classification Accuracy and Consistency of Binary Decisions," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 645-664, September.
    7. Giada Spaccapanico Proietti & Mariagiulia Matteucci & Stefania Mignani & Bernard P. Veldkamp, 2024. "Chance-Constrained Automated Test Assembly," Journal of Educational and Behavioral Statistics, , vol. 49(1), pages 92-120, February.
    8. Yang Liu & Jan Hannig & Abhishek Pal Majumder, 2019. "Second-Order Probability Matching Priors for the Person Parameter in Unidimensional IRT Models," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 701-718, September.
    9. Piero Veronese & Eugenio Melilli, 2021. "Confidence Distribution for the Ability Parameter of the Rasch Model," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 131-166, March.
    10. Yinhong He & Ping Chen, 2020. "Optimal Online Calibration Designs for Item Replenishment in Adaptive Testing," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 35-55, March.
    11. Yang Liu & Jan Hannig, 2017. "Generalized Fiducial Inference for Logistic Graded Response Models," Psychometrika, Springer;The Psychometric Society, vol. 82(4), pages 1097-1125, December.
    12. Jodi M. Casabianca & Charles Lewis, 2015. "IRT Item Parameter Recovery With Marginal Maximum Likelihood Estimation Using Loglinear Smoothing Models," Journal of Educational and Behavioral Statistics, , vol. 40(6), pages 547-578, December.
    13. Chun Wang, 2014. "Improving Measurement Precision of Hierarchical Latent Traits Using Adaptive Testing," Journal of Educational and Behavioral Statistics, , vol. 39(6), pages 452-477, December.
    14. Yang Liu & Jan Hannig, 2016. "Generalized Fiducial Inference for Binary Logistic Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 81(2), pages 290-324, June.
    15. Ionut Bebu & George Luta & Thomas Mathew & Brian K. Agan, 2016. "Generalized Confidence Intervals and Fiducial Intervals for Some Epidemiological Measures," IJERPH, MDPI, vol. 13(6), pages 1-13, June.
    16. Jinming Zhang, 2012. "The Impact of Variability of Item Parameter Estimators on Test Information Function," Journal of Educational and Behavioral Statistics, , vol. 37(6), pages 737-757, December.
    17. Kasy, Maximilian, 2011. "A nonparametric test for path dependence in discrete panel data," Economics Letters, Elsevier, vol. 113(2), pages 172-175.
    18. Ying Cheng & Ke-Hai Yuan, 2010. "The Impact of Fallible Item Parameter Estimates on Latent Trait Recovery," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 280-291, June.
    19. Acharki, Naoufal & Bertoncello, Antoine & Garnier, Josselin, 2023. "Robust prediction interval estimation for Gaussian processes by cross-validation method," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    20. Atı̇la Abdulkadı̇roğlu & Joshua D. Angrist & Yusuke Narita & Parag Pathak, 2022. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Econometrica, Econometric Society, vol. 90(1), pages 117-151, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:43:y:2018:i:3:p:259-285. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.