IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/xaqn6.html
   My bibliography  Save this paper

The PI: An alternative scoring algorithm for the IRAP using a probabilistic semiparametric effect measure

Author

Listed:
  • De Schryver, Maarten
  • Hussey, Ian

    (Ghent University)

  • De Neve, Jan
  • Cartwright, Aoife
  • Barnes-Holmes, Dermot

Abstract

The Implicit Relational Assessment Procedure (IRAP) has been used to assess the probability of arbitrarily applicable relational responding or as an indirect measure of implicit attitudes. To date, IRAP effects have commonly been quantified using the DIRAP scoring algorithm, which was derived from Greenwald, Nosek and Banaji’s (2003) D effect size measure. In the article, we highlight the difference between an effect size measure and a scoring algorithm, discuss the drawbacks associated with D, and propose an alternative: a probabilistic, semiparametric measure referred to as the Probabilistic Index (Thas, De Neve, Clement, & Ottoy, 2012). Using a relatively large IRAP dataset, we demonstrate how the PI is more robust to the influence of outliers and skew (which are typical of reaction time data), and improves internal consistency. Finally, we conclude that PI models, in addition to producing point estimate scores, can also provide confidence intervals, significance tests, and afford the possibility to include covariates, all of which may aid single subject design studies.

Suggested Citation

  • De Schryver, Maarten & Hussey, Ian & De Neve, Jan & Cartwright, Aoife & Barnes-Holmes, Dermot, 2017. "The PI: An alternative scoring algorithm for the IRAP using a probabilistic semiparametric effect measure," OSF Preprints xaqn6, Center for Open Science.
    • Handle: RePEc:osf:osfxxx:xaqn6
    DOI: 10.31219/osf.io/xaqn6
    as

    Download full text from publisher

    File URL: https://osf.io/download/5884e483594d9001f647f223/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/xaqn6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Peter Bentler, 2009. "Alpha, Dimension-Free, and Model-Based Internal Consistency Reliability," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 137-143, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Piotr Koc, 2021. "Measuring Non-electoral Political Participation: Bi-factor Model as a Tool to Extract Dimensions," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 156(1), pages 271-287, July.
    2. Yongxia Mei & Beilei Lin & Yingshuang Li & Chunge Ding & Zhenxiang Zhang, 2017. "Validity and reliability of Chinese version of Adult Carer Quality of Life questionnaire (AC-QoL) in family caregivers of stroke survivors," PLOS ONE, Public Library of Science, vol. 12(11), pages 1-14, November.
    3. Jeanne A. Teresi & Katja Ocepek-Welikson & John A. Toner & Marjorie Kleinman & Mildred Ramirez & Joseph P. Eimicke & Barry J. Gurland & Albert Siu, 2017. "Methodological Issues in Measuring Subjective Well-Being and Quality-of-Life: Applications to Assessment of Affect in Older, Chronically and Cognitively Impaired, Ethnically Diverse Groups Using the F," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 12(2), pages 251-288, June.
    4. Stephen R. Martin & Philippe Rast, 2022. "The Reliability Factor: Modeling Individual Reliability with Multiple Items from a Single Assessment," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1318-1342, December.
    5. Manuel García-Alonso & Miguel Ángel Gallardo-Vigil & Patricia Melgar Alcantud & Adrián Segura-Robles, 2020. "Social axioms on high school students in the North African context: Validation and fit of the SAS-II," PLOS ONE, Public Library of Science, vol. 15(11), pages 1-13, November.
    6. David J. Hessen, 2017. "Lower Bounds to the Reliabilities of Factor Score Estimators," Psychometrika, Springer;The Psychometric Society, vol. 82(3), pages 648-659, September.
    7. Kannan, Viji Diane & Brown, Theodore M. & Kunitz, Stephen J. & Chapman, Benjamin P., 2019. "Political parties and mortality: The role of social status and personal responsibility," Social Science & Medicine, Elsevier, vol. 223(C), pages 1-7.
    8. Jules L. Ellis, 2021. "A Test Can Have Multiple Reliabilities," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 869-876, December.
    9. Lindsey W. Vilca & Evelyn L. Chambi-Mamani & Emely D. Quispe-Kana & Mónica Hernández-López & Tomás Caycho-Rodríguez, 2022. "Functioning of the EROS-R Scale in a Clinical Sample of Psychiatric Patients: New Psychometric Evidence from the Classical Test Theory and the Item Response Theory," IJERPH, MDPI, vol. 19(16), pages 1-14, August.
    10. Peter M. Bentler, 2021. "Alpha, FACTT, and Beyond," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 861-868, December.
    11. David Gligor & Christopher Newman & Saim Kashmiri, 2021. "Does your skin color matter in buyer–seller negotiations? The implications of being a Black salesperson," Journal of the Academy of Marketing Science, Springer, vol. 49(5), pages 969-993, September.
    12. Tiina Latvala & Matthew Browne & Matthew Rockloff & Anne H. Salonen, 2021. "18-Item Version of the Short Gambling Harm Screen (SGHS-18): Validation of Screen for Assessing Gambling-Related Harm among Finnish Population," IJERPH, MDPI, vol. 18(21), pages 1-13, November.
    13. Maciej Koniewski & Ilona Barańska & Violetta Kijowska & Jenny T. Steen & Anne B. Wichmann & Sheila Payne & Giovanni Gambassi & Nele Den Noortgate & Harriet Finne-Soveri & Tinne Smets & Lieve den Block, 2022. "Measuring relatives’ perceptions of end-of-life communication with physicians in five countries: a psychometric analysis," European Journal of Ageing, Springer, vol. 19(4), pages 1561-1570, December.
    14. Peter M. Bentler, 2016. "Covariate-free and Covariate-dependent Reliability," Psychometrika, Springer;The Psychometric Society, vol. 81(4), pages 907-920, December.
    15. Klaas Sijtsma & Julius M. Pfadt, 2021. "Part II: On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha: Discussing Lower Bounds and Correlated Errors," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 843-860, December.
    16. Markus Pauly & Maria Umlauft & Ali Ünlü, 2018. "Resampling-Based Inference Methods for Comparing Two Coefficients Alpha," Psychometrika, Springer;The Psychometric Society, vol. 83(1), pages 203-222, March.
    17. Manohar, Sridhar & Paul, Justin & Strong, Carolyn & Mittal, Amit, 2023. "INNOSERV: Generalized scale for perceived service innovation," Journal of Business Research, Elsevier, vol. 160(C).
    18. Schroeders, Ulrich & Watrin, Luc & Wilhelm, Oliver, 2021. "Age-related nuances in knowledge assessment," Intelligence, Elsevier, vol. 85(C).
    19. Jiaxi Zhang & Meng Cui & Wei Wang & Huijie Lu & Qing Wu & Xia Zhu & Danmin Miao & Yan Zhang & Xi Feng & Wei Xiao, 2014. "The Coexistence of Coping Resources and Specific Coping Styles in Stress: Evidence from Full Information Item Bifactor Analysis," PLOS ONE, Public Library of Science, vol. 9(5), pages 1-11, May.
    20. Zhengguo Gu & Wilco H. M. Emons & Klaas Sijtsma, 2021. "Estimating Difference-Score Reliability in Pretest–Posttest Settings," Journal of Educational and Behavioral Statistics, , vol. 46(5), pages 592-610, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:xaqn6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.