IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/f4jrh.html
   My bibliography  Save this paper

Systematic Measurement Error Can be Detected and Quantified by Performing Ordinary Statistical Tests on Item Errors

Author

Listed:
  • Beatty, Michael
  • Siem, Faith
  • Brown, Craigory
  • Shenouda, Steven

Abstract

Self-report measures are ubiquitous in behavioral science and have been a key method used to examine evolutionary hypotheses. Indeed, evidence for the existence and functional boundaries of many psychological adaptations is inferred based on responses on self-report measures. Valid inferences based on self-report data require that questionnaire items measure only the construct they were designed to measure, which in statistical terms means that measurement error is predominately random. In contrast, systematic error indicates that items measure constructs in addition to those the instrument was designed to measure. Substantial systematic error associated with measures used in research risks theoretically consistent interpretations that are, instead, spurious effects of method error. Here, we propose a strategy for estimating the portion of systematic measurement error contained in multi-item self-report measures. This strategy rests on the observation that item-errors can be treated like any other set of scores and therefore have the potential to reveal otherwise unnoticed patterns in the data.

Suggested Citation

  • Beatty, Michael & Siem, Faith & Brown, Craigory & Shenouda, Steven, 2023. "Systematic Measurement Error Can be Detected and Quantified by Performing Ordinary Statistical Tests on Item Errors," OSF Preprints f4jrh, Center for Open Science.
    • Handle: RePEc:osf:osfxxx:f4jrh
    DOI: 10.31219/osf.io/f4jrh
    as

    Download full text from publisher

    File URL: https://osf.io/download/64e783b8939a2308357f725d/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/f4jrh?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Samuel Green & Yanyun Yang, 2009. "Commentary on Coefficient Alpha: A Cautionary Tale," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 121-135, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Klaas Sijtsma, 2012. "Future of Psychometrics: Ask What Psychometrics Can Do for Psychology," Psychometrika, Springer;The Psychometric Society, vol. 77(1), pages 4-20, January.
    2. Giacomo Angelini & Ilaria Buonomo & Paula Benevene & Piermarco Consiglio & Luciano Romano & Caterina Fiorilli, 2021. "The Burnout Assessment Tool (BAT): A Contribution to Italian Validation with Teachers’," Sustainability, MDPI, vol. 13(16), pages 1-18, August.
    3. Klaas Sijtsma & Julius M. Pfadt, 2021. "Part II: On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha: Discussing Lower Bounds and Correlated Errors," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 843-860, December.
    4. Markus Pauly & Maria Umlauft & Ali Ünlü, 2018. "Resampling-Based Inference Methods for Comparing Two Coefficients Alpha," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 203-222, March.
    5. Trinh Thi Thu Giang & John Andre & Ho Hoang Lan, 2022. "Student Engagement: Validating a Model to Unify In-Class and Out-of-Class Contexts," SAGE Open, , vol. 12(4), pages 21582440221, December.
    6. Ariel Alonso & Annouschka Laenen & Geert Molenberghs & Helena Geys & Tony Vangeneugden, 2010. "A Unified Approach to Multi-item Reliability," Biometrics, The International Biometric Society, vol. 66(4), pages 1061-1068, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:f4jrh. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.