IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v36y2011i6p736-754.html
   My bibliography  Save this article

Multimodal Likelihoods in Educational Assessment

Author

Listed:
  • Werner Wothke

    (The American Institutes for Research)

  • George Burket

    (Retired)

  • Li-Sue Chen

    (eBay)

  • Furong Gao

    (CTB/McGraw-Hill)

  • Lianghua Shu

    (CTB/McGraw-Hill)

  • Mike Chia

    (Consultant)

Abstract

It has been known for some time that item response theory (IRT) models may exhibit a likelihood function of a respondent’s ability which may have multiple modes, flat modes, or both. These conditions, often associated with guessing of multiple-choice (MC) questions, can introduce uncertainty and bias to ability estimation by maximum likelihood (ML) when standard Newton solutions are used. This article evaluates the performance of several maximization methods, including initial (grid) searches probing the function slopes, simulated annealing, exhaustive likelihood evaluation, and the standard Newton algorithm. In extensive studies, involving several million records of both generated and real data, the algorithms were evaluated with respect to precision and speed. Two methods, exhaustive search and grid search, followed by Newton steps, all yielded ML estimates at the required precision. At today’s computer speeds, either of these algorithms may be considered for high-volume response pattern scoring.

Suggested Citation

  • Werner Wothke & George Burket & Li-Sue Chen & Furong Gao & Lianghua Shu & Mike Chia, 2011. "Multimodal Likelihoods in Educational Assessment," Journal of Educational and Behavioral Statistics, , vol. 36(6), pages 736-754, December.
    • Handle: RePEc:sae:jedbes:v:36:y:2011:i:6:p:736-754
    DOI: 10.3102/1076998610381400
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998610381400
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998610381400?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Fumiko Samejima, 1973. "A comment on Birnbaum's three-parameter logistic model in the latent trait theory," Psychometrika, Springer;The Psychometric Society, vol. 38(2), pages 221-233, June.
    2. R. Bock & Murray Aitkin, 1981. "Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm," Psychometrika, Springer;The Psychometric Society, vol. 46(4), pages 443-459, December.
    3. Wendy Yen & George Burket & Robert Sykes, 1991. "Nonunique solutions to the likelihood equation for the three-parameter logistic model," Psychometrika, Springer;The Psychometric Society, vol. 56(1), pages 39-54, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Das, Jishnu & Zajonc, Tristan, 2010. "India shining and Bharat drowning: Comparing two Indian states to the worldwide distribution in mathematics achievement," Journal of Development Economics, Elsevier, vol. 92(2), pages 175-187, July.
    2. David Magis, 2015. "A Note on the Equivalence Between Observed and Expected Information Functions With Polytomous IRT Models," Journal of Educational and Behavioral Statistics, , vol. 40(1), pages 96-105, February.
    3. Elizabeth Ooi, 2020. "Give mind to the gap: Measuring gender differences in financial knowledge," Journal of Consumer Affairs, Wiley Blackwell, vol. 54(3), pages 931-950, September.
    4. Ying Cheng & Ke-Hai Yuan, 2010. "The Impact of Fallible Item Parameter Estimates on Latent Trait Recovery," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 280-291, June.
    5. Alexander Robitzsch, 2024. "Bias-Reduced Haebara and Stocking–Lord Linking," J, MDPI, vol. 7(3), pages 1-12, September.
    6. Joel A. Martínez-Regalado & Cinthia Leonora Murillo-Avalos & Purificación Vicente-Galindo & Mónica Jiménez-Hernández & José Luis Vicente-Villardón, 2021. "Using HJ-Biplot and External Logistic Biplot as Machine Learning Methods for Corporate Social Responsibility Practices for Sustainable Development," Mathematics, MDPI, vol. 9(20), pages 1-16, October.
    7. Vitoratou, Silia & Ntzoufras, Ioannis & Moustaki, Irini, 2016. "Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions," LSE Research Online Documents on Economics 57685, London School of Economics and Political Science, LSE Library.
    8. Michela Battauz & Ruggero Bellio, 2011. "Structural Modeling of Measurement Error in Generalized Linear Models with Rasch Measures as Covariates," Psychometrika, Springer;The Psychometric Society, vol. 76(1), pages 40-56, January.
    9. Melissa Gladstone & Gillian Lancaster & Gareth McCray & Vanessa Cavallera & Claudia R. L. Alves & Limbika Maliwichi & Muneera A. Rasheed & Tarun Dua & Magdalena Janus & Patricia Kariger, 2021. "Validation of the Infant and Young Child Development (IYCD) Indicators in Three Countries: Brazil, Malawi and Pakistan," IJERPH, MDPI, vol. 18(11), pages 1-19, June.
    10. Björn Andersson & Tao Xin, 2021. "Estimation of Latent Regression Item Response Theory Models Using a Second-Order Laplace Approximation," Journal of Educational and Behavioral Statistics, , vol. 46(2), pages 244-265, April.
    11. Norman Cliff, 1989. "Ordinal consistency and ordinal true scores," Psychometrika, Springer;The Psychometric Society, vol. 54(1), pages 75-91, March.
    12. Tim Kaiser & Luis Oberrauch & Günther Seeber, 2020. "Measuring economic competence of secondary school students in Germany," The Journal of Economic Education, Taylor & Francis Journals, vol. 51(3-4), pages 227-242, August.
    13. Eijffinger, Sylvester & Mahieu, Ronald & Raes, Louis, 2018. "Inferring hawks and doves from voting records," European Journal of Political Economy, Elsevier, vol. 51(C), pages 107-120.
    14. C. Glas & Anna Dagohoy, 2007. "A Person Fit Test For Irt Models For Polytomous Items," Psychometrika, Springer;The Psychometric Society, vol. 72(2), pages 159-180, June.
    15. Michael Edwards, 2010. "A Markov Chain Monte Carlo Approach to Confirmatory Item Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 75(3), pages 474-497, September.
    16. Yang Liu, 2020. "A Riemannian Optimization Algorithm for Joint Maximum Likelihood Estimation of High-Dimensional Exploratory Item Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 439-468, June.
    17. Cees Glas, 1988. "The derivation of some tests for the rasch model from the multinomial distribution," Psychometrika, Springer;The Psychometric Society, vol. 53(4), pages 525-546, December.
    18. Begoña A. Farizo & John Joyce & Mario Soliño, 2014. "Dealing with Heterogeneous Preferences Using Multilevel Mixed Models," Land Economics, University of Wisconsin Press, vol. 90(1), pages 181-198.
    19. Christian A. Gregory, 2020. "Are We Underestimating Food Insecurity? Partial Identification with a Bayesian 4-Parameter IRT Model," Journal of Classification, Springer;The Classification Society, vol. 37(3), pages 632-655, October.
    20. Javier Revuelta, 2004. "Analysis of distractor difficulty in multiple-choice items," Psychometrika, Springer;The Psychometric Society, vol. 69(2), pages 217-234, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:36:y:2011:i:6:p:736-754. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.