IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v45y2020i3p316-338.html
   My bibliography  Save this article

Adaptive Pairwise Comparison for Educational Measurement

Author

Listed:
  • Elise A. V. Crompvoets

    (Tilburg University
    Cito)

  • Anton A. Béguin

    (Cito)

  • Klaas Sijtsma

    (Tilburg University)

Abstract

Pairwise comparison is becoming increasingly popular as a holistic measurement method in education. Unfortunately, many comparisons are required for reliable measurement. To reduce the number of required comparisons, we developed an adaptive selection algorithm (ASA) that selects the most informative comparisons while taking the uncertainty of the object parameters into account. The results of the simulation study showed that, given the number of comparisons, the ASA resulted in smaller standard errors of object parameter estimates than a random selection algorithm that served as a benchmark. Rank order accuracy and reliability were similar for the two algorithms. Because the scale separation reliability (SSR) may overestimate the benchmark reliability when the ASA is used, caution is required when interpreting the SSR.

Suggested Citation

  • Elise A. V. Crompvoets & Anton A. Béguin & Klaas Sijtsma, 2020. "Adaptive Pairwise Comparison for Educational Measurement," Journal of Educational and Behavioral Statistics, , vol. 45(3), pages 316-338, June.
    • Handle: RePEc:sae:jedbes:v:45:y:2020:i:3:p:316-338
    DOI: 10.3102/1076998619890589
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998619890589
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998619890589?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Klaas Sijtsma, 2009. "On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 107-120, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chang, Long & Ma, Chen & Zhang, Chenghui & Duan, Bin & Cui, Naxin & Li, Changlong, 2023. "Correlations of lithium-ion battery parameter variations and connected configurations on pack statistics," Applied Energy, Elsevier, vol. 329(C).
    2. Hartman, Dana T. & Goodman, Gail S. & Grandchamp, Jaxon & Vidales, Daisy & Shaver, Phillip R. & Lawler, Michael J. & Hobbs, Sue D., 2023. "Outcomes for foster youth placed at a special residential high school: A quantitative case study," Children and Youth Services Review, Elsevier, vol. 151(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiaochuan Song, 2022. "Investigating Employees’ Responses to Abusive Supervision," Merits, MDPI, vol. 2(4), pages 1-20, November.
    2. Carmen León-Mantero & José Carlos Casas-Rosal & Alexander Maz-Machado & Miguel E Villarraga Rico, 2020. "Analysis of attitudinal components towards statistics among students from different academic degrees," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-13, January.
    3. Brian K Miller & Kay M Nicols & Silvia Clark & Alison Daniels & Whitney Grant, 2018. "Meta-analysis of coefficient alpha for scores on the Narcissistic Personality Inventory," PLOS ONE, Public Library of Science, vol. 13(12), pages 1-16, December.
    4. Adam Pawlewicz & Wojciech Gotkiewicz & Katarzyna Brodzińska & Katarzyna Pawlewicz & Bartosz Mickiewicz & Paweł Kluczek, 2022. "Organic Farming as an Alternative Maintenance Strategy in the Opinion of Farmers from Natura 2000 Areas," IJERPH, MDPI, vol. 19(7), pages 1-22, March.
    5. Michael Hennessy & Amy Bleakley & Martin Fishbein, 2012. "Measurement Models for Reasoned Action Theory," The ANNALS of the American Academy of Political and Social Science, , vol. 640(1), pages 42-57, March.
    6. Rauter, Romana & Globocnik, Dietfried & Baumgartner, Rupert J., 2023. "The role of organizational controls to advance sustainability innovation performance," Technovation, Elsevier, vol. 128(C).
    7. Mercy Gloria Ashepet & Liesbet Vranken & Caroline Michellier & Olivier Dewitte & Rodgers Mutyebere & Clovis Kabaseke & Ronald Twongyirwe & Violet Kanyiginya & Grace Kagoro-Rugunda & Tine Huyse & Liesb, 2024. "Assessing scale reliability in citizen science motivational research: lessons learned from two case studies in Uganda," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-18, December.
    8. Kelvin K. F. Law & Lillian F. Mills, 2019. "Financial Gatekeepers and Investor Protection: Evidence from Criminal Background Checks," Journal of Accounting Research, Wiley Blackwell, vol. 57(2), pages 491-543, May.
    9. Bastiaan Bruinsma, 2020. "A comparison of measures to validate scales in voting advice applications," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(4), pages 1299-1316, August.
    10. Wen Jiao & Peter Johannes Schulz & Angela Chang, 2024. "Addressing the role of eHealth literacy in shaping popular attitudes towards post-COVID-19 vaccination among Chinese adults," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-11, December.
    11. Jan-Benedict E. M. Steenkamp & Alberto Maydeu-Olivares, 2023. "Unrestricted factor analysis: A powerful alternative to confirmatory factor analysis," Journal of the Academy of Marketing Science, Springer, vol. 51(1), pages 86-113, January.
    12. King, Wesley M. & Hughto, Jaclyn M.W. & Operario, Don, 2020. "Transgender stigma: A critical scoping review of definitions, domains, and measures used in empirical research," Social Science & Medicine, Elsevier, vol. 250(C).
    13. Guido Alessandri & Michele Vecchione & Brent Donnellan & John Tisak, 2013. "An Application of the LC-LSTM Framework to the Self-esteem Instability Case," Psychometrika, Springer;The Psychometric Society, vol. 78(4), pages 769-792, October.
    14. Foltean, Florin Sabin & Trif, Simona Mihaela & Tuleu, Daniela Liliana, 2019. "Customer relationship management capabilities and social media technology use: Consequences on firm performance," Journal of Business Research, Elsevier, vol. 104(C), pages 563-575.
    15. Érika Martins Silva Ramos & Cecilia Jakobsson Bergstad, 2021. "The Psychology of Sharing: Multigroup Analysis among Users and Non-Users of Carsharing," Sustainability, MDPI, vol. 13(12), pages 1-17, June.
    16. István Tóth-Király & Beáta Bőthe & Gábor Orosz & Adrien Rigó, 2019. "A New Look on the Representation and Criterion Validity of Need Fulfillment: Application of the Bifactor Exploratory Structural Equation Modeling Framework," Journal of Happiness Studies, Springer, vol. 20(5), pages 1609-1626, June.
    17. Hermundur Sigmundsson & Håvard Lorås & Monika Haga, 2016. "Assessment of Motor Competence Across the Life Span," SAGE Open, , vol. 6(1), pages 21582440166, February.
    18. Ibrahim N. Khatatbeh & Hashem Alshurafat & Mohannad Obeid Al Shbail & Fouad Jamaani, 2023. "Factors Affecting Employees Use and Acceptance of Remote Working During the COVID-19 Pandemic: Evidence From the Jordanian Insurance Sector," SAGE Open, , vol. 13(2), pages 21582440231, June.
    19. Dorothy Watson & Bertrand Maitre, 2015. "Is Fuel Poverty in Ireland a Distinct Type of Deprivation?," The Economic and Social Review, Economic and Social Studies, vol. 46(2), pages 267-291.
    20. Li, Zhenhua & Gao, Xuan, 2021. "Makers’ relationship network, knowledge acquisition and innovation performance: An empirical analysis from China," Technology in Society, Elsevier, vol. 66(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:45:y:2020:i:3:p:316-338. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.