IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v111y2017i2d10.1007_s11192-017-2283-4.html
   My bibliography  Save this article

How to reduce the number of rating scale items without predictability loss?

Author

Listed:
  • W. W. Koczkodaj

    (Laurentian University)

  • T. Kakiashvili

    (Sudbury Therapy)

  • A. Szymańska

    (UKSW University)

  • J. Montero-Marin

    (University of Zaragoza)

  • R. Araya

    (London School of Hygiene and Tropical Medicine)

  • J. Garcia-Campayo

    (University of Zaragoza)

  • K. Rutkowski

    (Jagiellonian University)

  • D. Strzałka

    (Rzeszów University of Technology)

Abstract

Rating scales are used to elicit data about qualitative entities (e.g., research collaboration). This study presents an innovative method for reducing the number of rating scale items without the predictability loss. The “area under the receiver operator curve method” (AUC ROC) is used. The presented method has reduced the number of rating scale items (variables) to 28.57% (from 21 to 6) making over 70% of collected data unnecessary. Results have been verified by two methods of analysis: Graded Response Model (GRM) and Confirmatory Factor Analysis (CFA). GRM revealed that the new method differentiates observations of high and middle scores. CFA proved that the reliability of the rating scale has not deteriorated by the scale item reduction. Both statistical analysis evidenced usefulness of the AUC ROC reduction method.

Suggested Citation

  • W. W. Koczkodaj & T. Kakiashvili & A. Szymańska & J. Montero-Marin & R. Araya & J. Garcia-Campayo & K. Rutkowski & D. Strzałka, 2017. "How to reduce the number of rating scale items without predictability loss?," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(2), pages 581-593, May.
    • Handle: RePEc:spr:scient:v:111:y:2017:i:2:d:10.1007_s11192-017-2283-4
    DOI: 10.1007/s11192-017-2283-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-017-2283-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-017-2283-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rizopoulos, Dimitris, 2006. "ltm: An R Package for Latent Variable Modeling and Item Response Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 17(i05).
    2. Gualberto Buela-Casal & Izabela Zych, 2012. "What do the scientists think about the impact factor?," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 281-292, August.
    3. Philippe Moigne & Pascal Ragouet, 2012. "Science as instrumentation. The case for psychiatric rating scales," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 329-349, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Waldemar W. Koczkodaj & Mirosław Mazurek & Dominik Strzałka & Alicja Wolny-Dominiak & Marc Woodbury-Smith, 2019. "Electronic Health Record Breaches as Social Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(2), pages 861-871, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ramón A. Feenstra & Emilio Delgado López-Cózar, 2022. "Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2085-2103, April.
    2. Pantea Kamrani & Isabelle Dorsch & Wolfgang G. Stock, 2021. "Do researchers know what the h-index is? And how do they estimate its importance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5489-5508, July.
    3. Yang Yixin & Lü Xin & Ma Jian & Qiao Han, 2014. "A Robust Factor Analysis Model for Dichotomous Data," Journal of Systems Science and Information, De Gruyter, vol. 2(5), pages 437-450, October.
    4. Arulmani Thiyagarajan & Tyler G. James & Roy Rillera Marzo, 2022. "Psychometric properties of the 21-item Depression, Anxiety, and Stress Scale (DASS-21) among Malaysians during COVID-19: a methodological study," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-8, December.
    5. Cervantes, Víctor H., 2017. "DFIT: An R Package for Raju's Differential Functioning of Items and Tests Framework," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(i05).
    6. Andrew Karl & Randy Eubank & Jelena Milovanovic & Mark Reiser & Dennis Young, 2014. "Using RngStreams for parallel random number generation in C++ and R," Computational Statistics, Springer, vol. 29(5), pages 1301-1320, October.
    7. Sora Lee & Daniel M. Bolt, 2018. "Asymmetric Item Characteristic Curves and Item Complexity: Insights from Simulation and Real Data Analyses," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 453-475, June.
    8. Nick Bailey & Anne-Catherine Guio, 2022. "Adaptive Deprivation Scales in a Multi-National Context: The European Child Deprivation Indicators," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 15(6), pages 2335-2362, December.
    9. Andrei V. Grinëv & Daria S. Bylieva & Victoria V. Lobatyuk, 2021. "Russian University Teachers’ Perceptions of Scientometrics," Publications, MDPI, vol. 9(2), pages 1-16, May.
    10. Gustavo Dambiski Gomes de Carvalho & Luis Mauricio Martins de Resende & Joseane Pontes & Hélio Gomes de Carvalho & Leozenir Mendes Betim, 2021. "Innovation and Management in MSMEs: A Literature Review of Highly Cited Papers," SAGE Open, , vol. 11(4), pages 21582440211, October.
    11. Dawit G. Ayele & Temesgen Zewotir & Henry Mwambi, 2014. "Using Rasch Modeling to Re-Evaluate Rapid Malaria Diagnosis Test Analyses," IJERPH, MDPI, vol. 11(7), pages 1-11, June.
    12. Paula Fariña & Jorge González & Ernesto San Martín, 2019. "The Use of an Identifiability-Based Strategy for the Interpretation of Parameters in the 1PL-G and Rasch Models," Psychometrika, Springer;The Psychometric Society, vol. 84(2), pages 511-528, June.
    13. Simona Malovaná & Martin Hodula & Zuzana Gric, 2024. "Researching the Research: A Central Banking Edition," International Journal of Central Banking, International Journal of Central Banking, vol. 20(1), pages 263-323, February.
    14. Mark Alfano & Kathryn Iurino & Paul Stey & Brian Robinson & Markus Christen & Feng Yu & Daniel Lapsley, 2017. "Development and validation of a multi-dimensional measure of intellectual humility," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-28, August.
    15. Francisco José Eiroa-Orosa & Laura Limiñana-Bravo, 2019. "An Instrument to Measure Mental Health Professionals’ Beliefs and Attitudes towards Service Users’ Rights," IJERPH, MDPI, vol. 16(2), pages 1-16, January.
    16. Jinshu Cui & Heather Rosoff & Richard S. John, 2017. "A Polytomous Item Response Theory Model for Measuring Near-Miss Appraisal as a Psychological Trait," Decision Analysis, INFORMS, vol. 14(2), pages 75-86, June.
    17. Lindsey W. Vilca & Evelyn L. Chambi-Mamani & Emely D. Quispe-Kana & Mónica Hernández-López & Tomás Caycho-Rodríguez, 2022. "Functioning of the EROS-R Scale in a Clinical Sample of Psychiatric Patients: New Psychometric Evidence from the Classical Test Theory and the Item Response Theory," IJERPH, MDPI, vol. 19(16), pages 1-14, August.
    18. Fabrizio Maturo & Francesca Fortuna & Tonio Di Battista, 2019. "Testing Equality of Functions Across Multiple Experimental Conditions for Different Ability Levels in the IRT Context: The Case of the IPRASE TLT 2016 Survey," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 146(1), pages 19-39, November.
    19. Ellen Bernadette Maria Elsman & Gerardus Hermanus Maria Bartholomeus van Rens & Ruth Marie Antoinette van Nispen, 2018. "Psychometric properties of a new intake questionnaire for visually impaired young adults: The Participation and Activity Inventory for Young Adults (PAI-YA)," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-24, August.
    20. Sergio Copiello, 2020. "The alleged citation advantage of video abstracts may be a matter of self-citations and self-selection bias. Comment on “The impact of video abstract on citation counts” by Zong et al," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 751-757, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:111:y:2017:i:2:d:10.1007_s11192-017-2283-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.