IDEAS home Printed from https://ideas.repec.org/p/ehl/lserod/118223.html
   My bibliography  Save this paper

Model misspecification and robustness of observed-score test equating using propensity scores

Author

Listed:
  • Wallin, Gabriel
  • Wiberg, Marie

Abstract

This study explores the usefulness of covariates on equating test scores from nonequivalent test groups. The covariates are captured by an estimated propensity score, which is used as a proxy for latent ability to balance the test groups. The objective is to assess the sensitivity of the equated scores to various misspecifications in the propensity score model. The study assumes a parametric form of the propensity score and evaluates the effects of various misspecification scenarios on equating error. The results, based on both simulated and real testing data, show that (1) omitting an important covariate leads to biased estimates of the equated scores, (2) misspecifying a nonlinear relationship between the covariates and test scores increases the equating standard error in the tails of the score distributions, and (3) the equating estimators are robust against omitting a second-order term as well as using an incorrect link function in the propensity score estimation model. The findings demonstrate that auxiliary information is beneficial for test score equating in complex settings. However, it also sheds light on the challenge of making fair comparisons between nonequivalent test groups in the absence of common items. The study identifies scenarios, where equating performance is acceptable and problematic, provides practical guidelines, and identifies areas for further investigation.

Suggested Citation

  • Wallin, Gabriel & Wiberg, Marie, 2023. "Model misspecification and robustness of observed-score test equating using propensity scores," LSE Research Online Documents on Economics 118223, London School of Economics and Political Science, LSE Library.
  • Handle: RePEc:ehl:lserod:118223
    as

    Download full text from publisher

    File URL: http://eprints.lse.ac.uk/118223/
    File Function: Open access version.
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sandip Sinharay & Paul Holland, 2010. "The Missing Data Assumptions of the NEAT Design and their Implications for Test Equating," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 309-327, June.
    2. Gabriel Wallin & Marie Wiberg, 2019. "Kernel Equating Using Propensity Scores for Nonequivalent Groups," Journal of Educational and Behavioral Statistics, , vol. 44(4), pages 390-414, August.
    3. Alina Davier, 2013. "Observed-Score Equating: An Overview," Psychometrika, Springer;The Psychometric Society, vol. 78(4), pages 605-623, October.
    4. Andersson, Björn & Bränberg, Kenny & Wiberg, Marie, 2013. "Performing the Kernel Method of Test Equating with the Package kequate," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 55(i06).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gabriel Wallin & Marie Wiberg, 2019. "Kernel Equating Using Propensity Scores for Nonequivalent Groups," Journal of Educational and Behavioral Statistics, , vol. 44(4), pages 390-414, August.
    2. Albano, Anthony D., 2016. "equate: An R Package for Observed-Score Linking and Equating," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 74(i08).
    3. Björn Andersson & Marie Wiberg, 2017. "Item Response Theory Observed-Score Kernel Equating," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 48-66, March.
    4. Alina Davier, 2013. "Observed-Score Equating: An Overview," Psychometrika, Springer;The Psychometric Society, vol. 78(4), pages 605-623, October.
    5. González, Jorge & Barrientos, Andrés F. & Quintana, Fernando A., 2015. "Bayesian nonparametric estimation of test equating functions with covariates," Computational Statistics & Data Analysis, Elsevier, vol. 89(C), pages 222-244.
    6. González, Jorge, 2014. "SNSequate: Standard and Nonstandard Statistical Models and Methods for Test Equating," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 59(i07).
    7. Björn Andersson & Alina Davier, 2015. "Book Review," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 856-858, September.

    More about this item

    Keywords

    test score equating; kernel equating; nonequivalent groups; propensity scores; model misspecification;
    All these keywords.

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ehl:lserod:118223. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: LSERO Manager (email available below). General contact details of provider: https://edirc.repec.org/data/lsepsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.