IDEAS home Printed from https://ideas.repec.org/a/pal/palcom/v11y2024i1d10.1057_s41599-024-02873-1.html
   My bibliography  Save this article

Assessing scale reliability in citizen science motivational research: lessons learned from two case studies in Uganda

Author

Listed:
  • Mercy Gloria Ashepet

    (Royal Museum for Central Africa
    Department of Earth and Environmental Sciences
    Royal Museum for Central Africa)

  • Liesbet Vranken

    (Department of Earth and Environmental Sciences)

  • Caroline Michellier

    (Royal Museum for Central Africa)

  • Olivier Dewitte

    (Royal Museum for Central Africa)

  • Rodgers Mutyebere

    (Department of Earth and Environmental Sciences
    Mountains of the Moon University)

  • Clovis Kabaseke

    (Mountains of the Moon University)

  • Ronald Twongyirwe

    (Mbarara University of Science and Technology)

  • Violet Kanyiginya

    (Royal Museum for Central Africa
    Mbarara University of Science and Technology
    Vrije Universiteit Brussels)

  • Grace Kagoro-Rugunda

    (Mbarara University of Science and Technology)

  • Tine Huyse

    (Royal Museum for Central Africa)

  • Liesbet Jacobs

    (University of Amsterdam
    Department of Earth and Environmental Sciences)

Abstract

Citizen science (CS) is gaining global recognition for its potential to democratize and boost scientific research. As such, understanding why people contribute their time, energy, and skills to CS and why they (dis)continue their involvement is crucial. While several CS studies draw from existing theoretical frameworks in the psychology and volunteering fields to understand motivations, adapting these frameworks to CS research is still lagging and applications in the Global South remain limited. Here we investigated the reliability of two commonly applied psychometric tests, the Volunteer Functions Inventory (VFI) and the Theory of Planned Behaviour (TPB), to understand participant motivations and behaviour, in two CS networks in southwest Uganda, one addressing snail-borne diseases and another focused on natural hazards. Data was collected using a semi-structured questionnaire administered to the CS participants and a control group that consisted of candidate citizen scientists, under group and individual interview settings. Cronbach’s alpha, as an a priori measure of reliability, indicated moderate to low reliability for the VFI and TPB factors per CS network per interview setting. With evidence of highly skewed distributions, non-unidimensional data, correlated errors and lack of tau-equivalence, alpha’s underlying assumptions were often violated. More robust measures, McDonald’s omega and Greatest lower bound, generally showed higher reliability but confirmed overall patterns with VFI factors systematically scoring higher, and some TPB factors—perceived behavioural control, intention, self-identity, and moral obligation—scoring lower. Metadata analysis revealed that most problematic items often had weak item–total correlations. We propose that alpha should not be reported blindly without paying heed to the nature of the test, the assumptions, and the items comprising it. Additionally, we recommend caution when adopting existing theoretical frameworks to CS research and propose the development and validation of context-specific psychometric tests tailored to the unique CS landscape, especially for the Global South.

Suggested Citation

  • Mercy Gloria Ashepet & Liesbet Vranken & Caroline Michellier & Olivier Dewitte & Rodgers Mutyebere & Clovis Kabaseke & Ronald Twongyirwe & Violet Kanyiginya & Grace Kagoro-Rugunda & Tine Huyse & Liesb, 2024. "Assessing scale reliability in citizen science motivational research: lessons learned from two case studies in Uganda," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-18, December.
  • Handle: RePEc:pal:palcom:v:11:y:2024:i:1:d:10.1057_s41599-024-02873-1
    DOI: 10.1057/s41599-024-02873-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1057/s41599-024-02873-1
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1057/s41599-024-02873-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gerbing, David W & Anderson, James C, 1984. "On the Meaning of Within-Factor Correlated Measurement Errors," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 11(1), pages 572-580, June.
    2. Klaas Sijtsma, 2009. "On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 107-120, March.
    3. Yadanuch Boonyaratana & Eva Ekvall Hansson & Marianne Granbom & Steven M. Schmidt, 2021. "The Psychometric Properties of the Meaning of Home and Housing-Related Control Beliefs Scales among 67–70 Year-Olds in Sweden," IJERPH, MDPI, vol. 18(8), pages 1-12, April.
    4. Sauermann, Henry & Vohland, Katrin & Antoniou, Vyron & Balázs, Bálint & Göbel, Claudia & Karatzas, Kostas & Mooney, Peter & Perelló, Josep & Ponti, Marisa & Samson, Roeland & Winter, Silvia, 2020. "Citizen science and sustainability transitions," Research Policy, Elsevier, vol. 49(5).
    5. Jacobien Niebuur & Aart C. Liefbroer & Nardi Steverink & Nynke Smidt, 2019. "Translation and Validation of the Volunteer Functions Inventory (VFI) among the General Dutch Older Population," IJERPH, MDPI, vol. 16(17), pages 1-14, August.
    6. Mei‐Fang Chen, 2020. "The impacts of perceived moral obligation and sustainability self‐identity on sustainability development: A theory of planned behavior purchase intention model of sustainability‐labeled coffee and the," Business Strategy and the Environment, Wiley Blackwell, vol. 29(6), pages 2404-2417, September.
    7. Rosseel, Yves, 2012. "lavaan: An R Package for Structural Equation Modeling," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i02).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael Hennessy & Amy Bleakley & Martin Fishbein, 2012. "Measurement Models for Reasoned Action Theory," The ANNALS of the American Academy of Political and Social Science, , vol. 640(1), pages 42-57, March.
    2. Érika Martins Silva Ramos & Cecilia Jakobsson Bergstad, 2021. "The Psychology of Sharing: Multigroup Analysis among Users and Non-Users of Carsharing," Sustainability, MDPI, vol. 13(12), pages 1-17, June.
    3. André Pirralha & Wiebke Weber, 2020. "Correction for measurement error in invariance testing: An illustration using SQP," PLOS ONE, Public Library of Science, vol. 15(10), pages 1-18, October.
    4. Hartung, Johanna & Goecke, Benjamin & Schroeders, Ulrich & Schmitz, Florian & Wilhelm, Oliver, 2022. "Latin square tasks: A multi-study evaluation," Intelligence, Elsevier, vol. 94(C).
    5. Hernán Bejarano & Pedro Hancevic & Yadira Peralta, 2024. "Pro-Environmental Values and Energy Practices in Mexican Small and Medium-Sized Enterprises: Insights from the New Ecological Paradigm," Working Papers DTE 636, CIDE, División de Economía.
    6. Lena Busch & Till Utesch & Bernd Strauss, 2019. "Validation of the vignette-based German Exercise Causality Orientation Scale (G-ECOS)," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-20, October.
    7. Gordon W. Cheung & Helena D. Cooper-Thomas & Rebecca S. Lau & Linda C. Wang, 2024. "Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations," Asia Pacific Journal of Management, Springer, vol. 41(2), pages 745-783, June.
    8. Tormod Bøe & Mari Hysing & Jens Christoffer Skogen & Kyrre Breivik, 2016. "The Strengths and Difficulties Questionnaire (SDQ): Factor Structure and Gender Equivalence in Norwegian Adolescents," PLOS ONE, Public Library of Science, vol. 11(5), pages 1-15, May.
    9. Pascal Jordan & Meike C Shedden-Mora & Bernd Löwe, 2017. "Psychometric analysis of the Generalized Anxiety Disorder scale (GAD-7) in primary care using modern item response theory," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-14, August.
    10. Ulrich Schroeders & Oliver Wilhelm & Gabriel Olaru, 2016. "Meta-Heuristics in Short Scale Construction: Ant Colony Optimization and Genetic Algorithm," PLOS ONE, Public Library of Science, vol. 11(11), pages 1-19, November.
    11. Md. Mominur Rahman & Bilkis Akhter, 2021. "The impact of investment in human capital on bank performance: evidence from Bangladesh," Future Business Journal, Springer, vol. 7(1), pages 1-13, December.
    12. César Merino-Soto & Gina Chávez-Ventura & Verónica López-Fernández & Guillermo M. Chans & Filiberto Toledano-Toledano, 2022. "Learning Self-Regulation Questionnaire (SRQ-L): Psychometric and Measurement Invariance Evidence in Peruvian Undergraduate Students," Sustainability, MDPI, vol. 14(18), pages 1-17, September.
    13. Nathaniel Oliver Iotti & Damiano Menin & Tomas Jungert, 2022. "Early Adolescents’ Motivations to Defend Victims of Cyberbullying," IJERPH, MDPI, vol. 19(14), pages 1-9, July.
    14. Christoph Dworschak, 2024. "Bias mitigation in empirical peace and conflict studies: A short primer on posttreatment variables," Journal of Peace Research, Peace Research Institute Oslo, vol. 61(3), pages 462-476, May.
    15. Andreea-Ionela Puiu & Anca Monica Ardeleanu & Camelia Cojocaru & Anca Bratu, 2021. "Exploring the Effect of Status Quo, Innovativeness, and Involvement Tendencies on Luxury Fashion Innovations: The Mediation Role of Status Consumption," Mathematics, MDPI, vol. 9(9), pages 1-18, May.
    16. Slupphaug, KJell & Mehmetoglu, Mehmet & Mittner, Matthias, 2024. "modsem: An R package for estimating latent interactions and quadratic effects," OSF Preprints h3rpw, Center for Open Science.
    17. Allen, Jaime & Muñoz, Juan Carlos & Ortúzar, Juan de Dios, 2019. "On evasion behaviour in public transport: Dissatisfaction or contagion?," Transportation Research Part A: Policy and Practice, Elsevier, vol. 130(C), pages 626-651.
    18. Merkle, Edgar C. & Steyvers, Mark & Mellers, Barbara & Tetlock, Philip E., 2017. "A neglected dimension of good forecasting judgment: The questions we choose also matter," International Journal of Forecasting, Elsevier, vol. 33(4), pages 817-832.
    19. Sai-fu Fung & Esther Oi-wah Chow & Chau-kiu Cheung, 2020. "Development and Evaluation of the Psychometric Properties of a Brief Wisdom Development Scale," IJERPH, MDPI, vol. 17(8), pages 1-14, April.
    20. Goran Calic & Moren Lévesque & Anton Shevchenko, 2024. "On why women-owned businesses take more time to secure microloans," Small Business Economics, Springer, vol. 63(3), pages 917-938, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pal:palcom:v:11:y:2024:i:1:d:10.1057_s41599-024-02873-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: https://www.nature.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.