IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v46y2021i2p187-202.html
   My bibliography  Save this article

Aggregate-Level Test-Scale Linking: A New Solution for an Old Problem?

Author

Listed:
  • Tim Moses

    (102787College Board)

  • Neil J. Dorans

    (6729Educational Testing Service)

Abstract

The Reardon, Kalogrides, and Ho article on validation methods for aggregate-level test scale linking is an attempt to validate a district-level scale aligning procedure that appears to be a new solution to an old problem. Their aligning procedure uses the National Assessment of Educational Progress (NAEP) scale to piece together a patchwork of data structures from different tests of different constructs obtained under different administration conditions and used in different ways by different states. In this article, we critique their linking and validation efforts. Our critique has three components. First, we review the recommendations for linking state assessments to NAEP from several studies and commentaries to provide background from which to interpret Reardon et al.’s validation attempts. Second, we provide a replication of the Reardon et al. empirical validations of its proposed linking procedure to demonstrate that correlations between district means on two test scores can be high even when (1) the constructs being measured by the tests are different and (2) the district-level means estimated using the Reardon et al. linking approach can differ substantially from actual district-level means. Then, we suggest additional checks for construct similarity and subpopulation invariance from other concordance studies that could be used to assess whether the inferences made by Reardon et al. are warranted. Finally, until such checks are made, we urge cautious use of the results of the Reardon et al. results.

Suggested Citation

  • Tim Moses & Neil J. Dorans, 2021. "Aggregate-Level Test-Scale Linking: A New Solution for an Old Problem?," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 187-202, April.
  • Handle: RePEc:sae:jedbes:v:46:y:2021:i:2:p:187-202
    DOI: 10.3102/1076998620960089
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998620960089
    Download Restriction: no

    File URL: https://libkey.io/10.3102/1076998620960089?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Andrew D. Ho & Sean F. Reardon & Demetra Kalogrides, 2021. "Validation Methods for Aggregate-Level Test Scale Linking: A Rejoinder," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 209-218, April.
    2. Sean F. Reardon & Demetra Kalogrides & Andrew D. Ho, 2021. "Validation Methods for Aggregate-Level Test Scale Linking: A Case Study Mapping School District Test Score Distributions to a Common Scale," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 138-167, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andrew D. Ho & Sean F. Reardon & Demetra Kalogrides, 2021. "Validation Methods for Aggregate-Level Test Scale Linking: A Rejoinder," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 209-218, April.
    2. Daniel F. McCaffrey & Steven A. Culpepper, 2021. "Introduction to JEBS Special Issue on NAEP Linked Aggregate Scores," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 135-137, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mark L. Davison, 2021. "Commentary on “Validation Methods for Aggregate-Level Test Scale Linking: A Case Study Mapping School District Test Score Distributions to a Common Scaleâ€," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 173-186, April.
    2. Daniel F. McCaffrey & Steven A. Culpepper, 2021. "Introduction to JEBS Special Issue on NAEP Linked Aggregate Scores," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 135-137, April.
    3. Andrew D. Ho & Sean F. Reardon & Demetra Kalogrides, 2021. "Validation Methods for Aggregate-Level Test Scale Linking: A Rejoinder," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 209-218, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:46:y:2021:i:2:p:187-202. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.