IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v87y2022i4d10.1007_s11336-022-09843-z.html
   My bibliography  Save this article

On the Information Obtainable from Comparative Judgments

Author

Listed:
  • Paul-Christian Bürkner

    (University of Stuttgart)

Abstract

Personality tests employing comparative judgments have been proposed as an alternative to Likert-type rating scales. One of the main advantages of a comparative format is that it can reduce faking of responses in high-stakes situations. However, previous research has shown that it is highly difficult to obtain trait score estimates that are both faking resistant and sufficiently accurate for individual-level diagnostic decisions. With the goal of contributing to a solution, I study the information obtainable from comparative judgments analyzed by means of Thurstonian IRT models. First, I extend the mathematical theory of ordinal comparative judgments and corresponding models. Second, I provide optimal test designs for Thurstonian IRT models that maximize the accuracy of people’s trait score estimates from both frequentist and Bayesian statistical perspectives. Third, I derive analytic upper bounds for the accuracy of these trait estimates achievable through ordinal Thurstonian IRT models. Fourth, I perform numerical experiments that complement results obtained in earlier simulation studies. The combined analytical and numerical results suggest that it is indeed possible to design personality tests using comparative judgments that yield trait scores estimates sufficiently accurate for individual-level diagnostic decisions, while reducing faking in high-stakes situations. Recommendations for the practical application of comparative judgments for the measurement of personality, specifically in high-stakes situations, are given.

Suggested Citation

  • Paul-Christian Bürkner, 2022. "On the Information Obtainable from Comparative Judgments," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1439-1472, December.
    • Handle: RePEc:spr:psycho:v:87:y:2022:i:4:d:10.1007_s11336-022-09843-z
    DOI: 10.1007/s11336-022-09843-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-022-09843-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-022-09843-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dennis Schmidt & Rainer Schwabe, 2015. "On optimal designs for censored data," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 78(3), pages 237-257, April.
    2. Anna Brown, 2016. "Item Response Models for Forced-Choice Questionnaires: A Common Framework," Psychometrika, Springer;The Psychometric Society, vol. 81(1), pages 135-160, March.
    3. Dennis Schmidt & Rainer Schwabe, 2015. "Erratum to: On optimal designs for censored data," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 78(3), pages 259-259, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Qian Wu & Monique Vanerum & Anouk Agten & Andrés Christiansen & Frank Vandenabeele & Jean-Michel Rigo & Rianne Janssen, 2021. "Certainty-Based Marking on Multiple-Choice Items: Psychometrics Meets Decision Theory," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 518-543, June.
    2. Alwyn Lim & Shawn Pope, 2022. "What drives companies to do good? A “universal” ordering of corporate social responsibility motivations," Corporate Social Responsibility and Environmental Management, John Wiley & Sons, vol. 29(1), pages 233-255, January.
    3. Susanne Frick, 2022. "Modeling Faking in the Multidimensional Forced-Choice Format: The Faking Mixture Model," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 773-794, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:87:y:2022:i:4:d:10.1007_s11336-022-09843-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.