IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v42y2017i1p3-45.html
   My bibliography  Save this article

Using Heteroskedastic Ordered Probit Models to Recover Moments of Continuous Test Score Distributions From Coarsened Data

Author

Listed:
  • Sean F. Reardon
  • Benjamin R. Shear

    (Stanford University)

  • Katherine E. Castellano

    (Educational Testing Service)

  • Andrew D. Ho

    (Harvard Graduate School of Education)

Abstract

Test score distributions of schools or demographic groups are often summarized by frequencies of students scoring in a small number of ordered proficiency categories. We show that heteroskedastic ordered probit (HETOP) models can be used to estimate means and standard deviations of multiple groups’ test score distributions from such data. Because the scale of HETOP estimates is indeterminate up to a linear transformation, we develop formulas for converting the HETOP parameter estimates and their standard errors to a scale in which the population distribution of scores is standardized. We demonstrate and evaluate this novel application of the HETOP model with a simulation study and using real test score data from two sources. We find that the HETOP model produces unbiased estimates of group means and standard deviations, except when group sample sizes are small. In such cases, we demonstrate that a “partially heteroskedastic†ordered probit (PHOP) model can produce estimates with a smaller root mean squared error than the fully heteroskedastic model.

Suggested Citation

  • Sean F. Reardon & Benjamin R. Shear & Katherine E. Castellano & Andrew D. Ho, 2017. "Using Heteroskedastic Ordered Probit Models to Recover Moments of Continuous Test Score Distributions From Coarsened Data," Journal of Educational and Behavioral Statistics, , vol. 42(1), pages 3-45, February.
    • Handle: RePEc:sae:jedbes:v:42:y:2017:i:1:p:3-45
    DOI: 10.3102/1076998616666279
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998616666279
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998616666279?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Greene,William H. & Hensher,David A., 2010. "Modeling Ordered Choices," Cambridge Books, Cambridge University Press, number 9780521194204, November.
    2. Joel L. Horowitz & Jürg M. Sparmann & Carlos F. Daganzo, 1982. "An Investigation of the Accuracy of the Clark Approximation for the Multinomial Probit Model," Transportation Science, INFORMS, vol. 16(3), pages 382-401, August.
    3. Keane, Michael P, 1992. "A Note on Identification in the Multinomial Probit Model," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(2), pages 193-200, April.
    4. Yuanyuan Gu & Denzil G. Fiebig & Edward Cripps & Robert Kohn, 2009. "Bayesian estimation of a random effects heteroscedastic probit model," Econometrics Journal, Royal Economic Society, vol. 12(2), pages 324-339, July.
    5. Anna N. Angelos Tosteson & Colin B. Begg, 1988. "A General Regression Methodology for ROC Curve Estimation," Medical Decision Making, , vol. 8(3), pages 204-215, August.
    6. Richard Williams, 2010. "Fitting heterogeneous choice models with oglm," Stata Journal, StataCorp LLC, vol. 10(4), pages 540-567, December.
    7. Alexander Shapiro & Jos Berge, 2002. "Statistical inference of minimum rank factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 67(1), pages 79-94, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Charlie Tchinda & Marcus Dejardin, 2021. "Are Business Policy Measures in Response to the COVID-19 Pandemic to Be Equally Valued? An Exploration According to SMEs Owners’ Business Expectations," Sustainability, MDPI, vol. 13(21), pages 1-42, October.
    2. Chen, Le-Yu & Oparina, Ekaterina & Powdthavee, Nattavudh & Srisuma, Sorawoot, 2022. "Robust Ranking of Happiness Outcomes: A Median Regression Perspective," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 672-686.
    3. Paul Gertler & Roland Sturm & Bruce Davidson, 1994. "Information and the Demand for Supplemental Medicare Insurance," NBER Working Papers 4700, National Bureau of Economic Research, Inc.
    4. Andrew S. Fullerton & Jun Xu, 2018. "Constrained and Unconstrained Partial Adjacent Category Logit Models for Ordinal Response Variables," Sociological Methods & Research, , vol. 47(2), pages 169-206, March.
    5. Chen, Le-Yu & Oparina, Ekaterina & Powdthavee, Nattavudh & Srisuma, Sorawoot, 2019. "Have Econometric Analyses of Happiness Data Been Futile? A Simple Truth about Happiness Scales," IZA Discussion Papers 12152, Institute of Labor Economics (IZA).
    6. Juan Carlos Martín & Concepción Román, 2021. "COVID-19 Is Examining the EU and the Member States: The Role of Attitudes and Sociodemographic Factors on Citizens’ Support towards National Policies," Social Sciences, MDPI, vol. 10(2), pages 1-27, January.
    7. Omar Paccagnella, 2011. "Anchoring vignettes with sample selection due to non‐response," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(3), pages 665-687, July.
    8. Benjamin R. Shear & Sean F. Reardon, 2021. "Using Pooled Heteroskedastic Ordered Probit Models to Improve Small-Sample Estimates of Latent Test Score Distributions," Journal of Educational and Behavioral Statistics, , vol. 46(1), pages 3-33, February.
    9. Nowak, Piotr Bolesław, 2016. "The MLE of the mean of the exponential distribution based on grouped data is stochastically increasing," Statistics & Probability Letters, Elsevier, vol. 111(C), pages 49-54.
    10. Fernandez-Cornejo, Jorge & Wechsler, Seth James, 2012. "Fifteen Years Later: Examining the Adoption of Bt Corn Varieties by U.S. Farmers," 2012 Annual Meeting, August 12-14, 2012, Seattle, Washington 124257, Agricultural and Applied Economics Association.
    11. Matthew Gentzkow, 2007. "Valuing New Goods in a Model with Complementarity: Online Newspapers," American Economic Review, American Economic Association, vol. 97(3), pages 713-744, June.
    12. Dixon, Huw D. & Grimme, Christian, 2022. "State-dependent or time-dependent pricing? New evidence from a monthly firm-level survey: 1980–2017," European Economic Review, Elsevier, vol. 150(C).
    13. An, Wookhyun & Alarcón, Silverio, 2021. "Rural tourism preferences in Spain: Best-worst choices," Annals of Tourism Research, Elsevier, vol. 89(C).
    14. Stefan Boes, 2013. "Nonparametric analysis of treatment effects in ordered response models," Empirical Economics, Springer, vol. 44(1), pages 81-109, February.
    15. Haaijer, Marinus E., 1996. "Predictions in conjoint choice experiments : the x-factor probit model," Research Report 96B22, University of Groningen, Research Institute SOM (Systems, Organisations and Management).
    16. William H. Greene & Mark N. Harris & Rachel J. Knott & Nigel Rice, 2021. "Specification and testing of hierarchical ordered response models with anchoring vignettes," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 31-64, January.
    17. Camilo Alberto Cárdenas-Hurtado & Aaron Levi Garavito-Acosta & Jorge Hernán Toro-Córdoba, 2018. "Asymmetric Effects of Terms of Trade Shocks on Tradable and Non-tradable Investment Rates: The Colombian Case," Borradores de Economia 1043, Banco de la Republica de Colombia.
    18. Yai, Tetsuo & Iwakura, Seiji & Morichi, Shigeru, 1997. "Multinomial probit with structured covariance for route choice behavior," Transportation Research Part B: Methodological, Elsevier, vol. 31(3), pages 195-207, June.
    19. Hanna Dudek & Joanna Landmesser, 2012. "Income satisfaction and relative deprivation," Statistics in Transition new series, Główny Urząd Statystyczny (Polska), vol. 13(2), pages 321-334, June.
    20. Das, Ujjwal & Das, Kalyan, 2018. "Inference on zero inflated ordinal models with semiparametric link," Computational Statistics & Data Analysis, Elsevier, vol. 128(C), pages 104-115.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:42:y:2017:i:1:p:3-45. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.