IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0167110.html
   My bibliography  Save this article

Meta-Heuristics in Short Scale Construction: Ant Colony Optimization and Genetic Algorithm

Author

Listed:
  • Ulrich Schroeders
  • Oliver Wilhelm
  • Gabriel Olaru

Abstract

The advent of large-scale assessment, but also the more frequent use of longitudinal and multivariate approaches to measurement in psychological, educational, and sociological research, caused an increased demand for psychometrically sound short scales. Shortening scales economizes on valuable administration time, but might result in inadequate measures because reducing an item set could: a) change the internal structure of the measure, b) result in poorer reliability and measurement precision, c) deliver measures that cannot effectively discriminate between persons on the intended ability spectrum, and d) reduce test-criterion relations. Different approaches to abbreviate measures fare differently with respect to the above-mentioned problems. Therefore, we compare the quality and efficiency of three item selection strategies to derive short scales from an existing long version: a Stepwise COnfirmatory Factor Analytical approach (SCOFA) that maximizes factor loadings and two metaheuristics, specifically an Ant Colony Optimization (ACO) with a tailored user-defined optimization function and a Genetic Algorithm (GA) with an unspecific cost-reduction function. SCOFA compiled short versions were highly reliable, but had poor validity. In contrast, both metaheuristics outperformed SCOFA and produced efficient and psychometrically sound short versions (unidimensional, reliable, sensitive, and valid). We discuss under which circumstances ACO and GA produce equivalent results and provide recommendations for conditions in which it is advisable to use a metaheuristic with an unspecific out-of-the-box optimization function.

Suggested Citation

  • Ulrich Schroeders & Oliver Wilhelm & Gabriel Olaru, 2016. "Meta-Heuristics in Short Scale Construction: Ant Colony Optimization and Genetic Algorithm," PLOS ONE, Public Library of Science, vol. 11(11), pages 1-19, November.
  • Handle: RePEc:plo:pone00:0167110
    DOI: 10.1371/journal.pone.0167110
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0167110
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0167110&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0167110?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Harold Gulliksen, 1945. "The relation of item difficulty and inter-item correlation to test variance and reliability," Psychometrika, Springer;The Psychometric Society, vol. 10(2), pages 79-91, June.
    2. Denny Borsboom, 2006. "The attack of the psychometricians," Psychometrika, Springer;The Psychometric Society, vol. 71(3), pages 425-440, September.
    3. Rosseel, Yves, 2012. "lavaan: An R Package for Structural Equation Modeling," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i02).
    4. Yutaka Kano & Akira Harada, 2000. "Stepwise variable selection in factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 65(1), pages 7-22, March.
    5. Klaas Sijtsma, 2009. "On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 107-120, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sai-fu Fung & Esther Oi-wah Chow & Chau-kiu Cheung, 2020. "Development and Evaluation of the Psychometric Properties of a Brief Wisdom Development Scale," IJERPH, MDPI, vol. 17(8), pages 1-14, April.
    2. Sam S. S. Lau & Cherry C. Y. Ho & Rebecca C. K. Pang & Susan Su & Heather Kwok & Sai-fu Fung & Roger C. Ho, 2022. "COVID-19 Burnout Subject to the Dynamic Zero-COVID Policy in Hong Kong: Development and Psychometric Evaluation of the COVID-19 Burnout Frequency Scale," Sustainability, MDPI, vol. 14(14), pages 1-13, July.
    3. Steger, Diana & Schroeders, Ulrich & Wilhelm, Oliver, 2019. "On the dimensionality of crystallized intelligence: A smartphone-based assessment," Intelligence, Elsevier, vol. 72(C), pages 76-85.
    4. Dennis Sing-wing Wong & Sai-fu Fung, 2020. "Development of the Cybercrime Rapid Identification Tool for Adolescents," IJERPH, MDPI, vol. 17(13), pages 1-13, June.
    5. Tifferet, Sigal, 2021. "Verifying online information: Development and validation of a self-report scale," Technology in Society, Elsevier, vol. 67(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pascal Jordan & Meike C Shedden-Mora & Bernd Löwe, 2017. "Psychometric analysis of the Generalized Anxiety Disorder scale (GAD-7) in primary care using modern item response theory," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-14, August.
    2. Mercy Gloria Ashepet & Liesbet Vranken & Caroline Michellier & Olivier Dewitte & Rodgers Mutyebere & Clovis Kabaseke & Ronald Twongyirwe & Violet Kanyiginya & Grace Kagoro-Rugunda & Tine Huyse & Liesb, 2024. "Assessing scale reliability in citizen science motivational research: lessons learned from two case studies in Uganda," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-18, December.
    3. Lena Busch & Till Utesch & Bernd Strauss, 2019. "Validation of the vignette-based German Exercise Causality Orientation Scale (G-ECOS)," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-20, October.
    4. Tormod Bøe & Mari Hysing & Jens Christoffer Skogen & Kyrre Breivik, 2016. "The Strengths and Difficulties Questionnaire (SDQ): Factor Structure and Gender Equivalence in Norwegian Adolescents," PLOS ONE, Public Library of Science, vol. 11(5), pages 1-15, May.
    5. Érika Martins Silva Ramos & Cecilia Jakobsson Bergstad, 2021. "The Psychology of Sharing: Multigroup Analysis among Users and Non-Users of Carsharing," Sustainability, MDPI, vol. 13(12), pages 1-17, June.
    6. Myszkowski, Nils & Storme, Martin, 2018. "A snapshot of g? Binary and polytomous item-response theory investigations of the last series of the Standard Progressive Matrices (SPM-LS)," Intelligence, Elsevier, vol. 68(C), pages 109-116.
    7. Hartung, Johanna & Goecke, Benjamin & Schroeders, Ulrich & Schmitz, Florian & Wilhelm, Oliver, 2022. "Latin square tasks: A multi-study evaluation," Intelligence, Elsevier, vol. 94(C).
    8. Hernán Bejarano & Pedro Hancevic & Yadira Peralta, 2024. "Pro-Environmental Values and Energy Practices in Mexican Small and Medium-Sized Enterprises: Insights from the New Ecological Paradigm," Working Papers DTE 636, CIDE, División de Economía.
    9. Sonia Nawrocka & Hans De Witte & Margherita Pasini & Margherita Brondino, 2023. "A Person-Centered Approach to Job Insecurity: Is There a Reciprocal Relationship between the Quantitative and Qualitative Dimensions of Job Insecurity?," IJERPH, MDPI, vol. 20(7), pages 1-27, March.
    10. Md. Mominur Rahman & Bilkis Akhter, 2021. "The impact of investment in human capital on bank performance: evidence from Bangladesh," Future Business Journal, Springer, vol. 7(1), pages 1-13, December.
    11. Masashi Soga & Kevin J. Gaston & Yuichi Yamaura & Kiyo Kurisu & Keisuke Hanaki, 2016. "Both Direct and Vicarious Experiences of Nature Affect Children’s Willingness to Conserve Biodiversity," IJERPH, MDPI, vol. 13(6), pages 1-12, May.
    12. César Merino-Soto & Gina Chávez-Ventura & Verónica López-Fernández & Guillermo M. Chans & Filiberto Toledano-Toledano, 2022. "Learning Self-Regulation Questionnaire (SRQ-L): Psychometric and Measurement Invariance Evidence in Peruvian Undergraduate Students," Sustainability, MDPI, vol. 14(18), pages 1-17, September.
    13. Steven Andrew Culpepper & Herman Aguinis & Justin L. Kern & Roger Millsap, 2019. "High-Stakes Testing Case Study: A Latent Variable Approach for Assessing Measurement and Prediction Invariance," Psychometrika, Springer;The Psychometric Society, vol. 84(1), pages 285-309, March.
    14. Nathaniel Oliver Iotti & Damiano Menin & Tomas Jungert, 2022. "Early Adolescents’ Motivations to Defend Victims of Cyberbullying," IJERPH, MDPI, vol. 19(14), pages 1-9, July.
    15. AJ Golio, 2024. "Whose Neighborhood Now? Gentrification and Community Life in Low-Income Urban Neighborhoods," Working Papers 24-29, Center for Economic Studies, U.S. Census Bureau.
    16. Peter Tavel & Bibiana Jozefiakova & Peter Telicak & Jana Furstova & Michal Puza & Natalia Kascakova, 2022. "Psychometric Analysis of the Shortened Version of the Spiritual Well-Being Scale on the Slovak Population (SWBS-Sk)," IJERPH, MDPI, vol. 19(1), pages 1-12, January.
    17. Allen, Jaime & Eboli, Laura & Forciniti, Carmen & Mazzulla, Gabriella & Ortúzar, Juan de Dios, 2019. "The role of critical incidents and involvement in transit satisfaction and loyalty," Transport Policy, Elsevier, vol. 75(C), pages 57-69.
    18. Christoph Dworschak, 2024. "Bias mitigation in empirical peace and conflict studies: A short primer on posttreatment variables," Journal of Peace Research, Peace Research Institute Oslo, vol. 61(3), pages 462-476, May.
    19. Andreea-Ionela Puiu & Anca Monica Ardeleanu & Camelia Cojocaru & Anca Bratu, 2021. "Exploring the Effect of Status Quo, Innovativeness, and Involvement Tendencies on Luxury Fashion Innovations: The Mediation Role of Status Consumption," Mathematics, MDPI, vol. 9(9), pages 1-18, May.
    20. Slupphaug, KJell & Mehmetoglu, Mehmet & Mittner, Matthias, 2024. "modsem: An R package for estimating latent interactions and quadratic effects," OSF Preprints h3rpw, Center for Open Science.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0167110. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.