IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v128y2023i1d10.1007_s11192-022-04542-z.html
   My bibliography  Save this article

Evaluating the online impact of reporting guidelines for randomised trial reports and protocols: a cross-sectional web-based data analysis of CONSORT and SPIRIT initiatives

Author

Listed:
  • Enrique Orduña-Malea

    (Universitat Politècnica de València)

  • Adolfo Alonso-Arroyo

    (University of Valencia
    UISYS)

  • José-Antonio Ontalba-Ruipérez

    (Universitat Politècnica de València)

  • Ferrán Catalá-López

    (National School of Public Health, Institute of Health Carlos III
    University of Valencia/INCLIVA Health Research Institute and CIBERSAM
    Ottawa Hospital Research Institute)

Abstract

Reporting guidelines are tools to help improve the transparency, completeness, and clarity of published articles in health research. Specifically, the CONSORT (Consolidated Standards of Reporting Trials) and SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) statements provide evidence-based guidance on what to include in randomised trial articles and protocols to guarantee the efficacy of interventions. These guidelines are subsequently described and discussed in journal articles and used to produce checklists. Determining the online impact (i.e., number and type of links received) of these articles can provide insights into the dissemination of reporting guidelines in broader environments (web-at-large) than simply that of the scientific publications that cite them. To address the technical limitations of link analysis, here the Debug-Validate-Access-Find (DVAF) method is designed and implemented to measure different facets of the guidelines’ online impact. A total of 65 articles related to 38 reporting guidelines are taken as a baseline, providing 240,128 URL citations, which are then refined, analysed, and categorised using the DVAF method. A total of 15,582 links to journal articles related to the CONSORT and SPIRIT initiatives were identified. CONSORT 2010 and SPIRIT 2013 were the reporting guidelines that received most links (URL citations) from other online objects (5328 and 2190, respectively). Overall, the online impact obtained is scattered (URL citations are received by different article URL IDs, mainly from link-based DOIs), narrow (limited number of linking domain names, half of articles are linked from fewer than 29 domain names), concentrated (links come from just a few academic publishers, around 60% from publishers), non-reputed (84% of links come from dubious websites and fake domain names) and highly decayed (89% of linking domain names were not accessible at the time of the analysis). In light of these results, it is concluded that the online impact of these guidelines could be improved, and a set of recommendations are proposed to this end.

Suggested Citation

  • Enrique Orduña-Malea & Adolfo Alonso-Arroyo & José-Antonio Ontalba-Ruipérez & Ferrán Catalá-López, 2023. "Evaluating the online impact of reporting guidelines for randomised trial reports and protocols: a cross-sectional web-based data analysis of CONSORT and SPIRIT initiatives," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 407-440, January.
    • Handle: RePEc:spr:scient:v:128:y:2023:i:1:d:10.1007_s11192-022-04542-z
    DOI: 10.1007/s11192-022-04542-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-022-04542-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-022-04542-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mike Thelwall & Nabeil Maflahi, 2016. "Guideline references and academic citations as evidence of the clinical value of health research," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(4), pages 960-966, April.
    2. Enrique Orduña-Malea & Rodrigo Costas, 2021. "Link-based approach to study scientific software usage: the case of VOSviewer," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 8153-8186, September.
    3. Wallace Koehler, 1999. "An analysis of web page and web site constancy and permanence," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 50(2), pages 162-180.
    4. Franz Barjak & Mike Thelwall, 2008. "A statistical analysis of the web presences of European life sciences research teams," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(4), pages 628-643, February.
    5. Isidro F. Aguillo & José L. Ortega & Mario Fernández & Ana M. Utrilla, 2010. "Indicators for a webometric ranking of open access repositories," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 477-486, March.
    6. Nigel Payne & Mike Thelwall, 2007. "A longitudinal study of academic webs: Growth and stabilisation," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(3), pages 523-539, June.
    7. Enrique Orduna-Malea & Mike Thelwall & Kayvan Kousha, 2017. "Web citations in patents: Evidence of technological impact?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(8), pages 1967-1974, August.
    8. Mousumi Karmakar & Sumit Kumar Banshal & Vivek Kumar Singh, 2021. "A large-scale comparison of coverage and mentions captured by the two altmetric aggregators: Altmetric.com and PlumX," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4465-4489, May.
    9. José Luis Ortega, 2018. "Reliability and accuracy of altmetric providers: a comparison among Altmetric.com, PlumX and Crossref Event Data," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2123-2138, September.
    10. Liwen Vaughan & Mike Thelwall, 2003. "Scholarly use of the Web: What are the key inducers of links to journal Web sites?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 54(1), pages 29-38, January.
    11. Font-Julián, Cristina I & Ontalba-Ruipérez, José-Antonio & Orduña-Malea, Enrique & Thelwall, Mike, 2022. "Which types of online resource support US patent claims?," Journal of Informetrics, Elsevier, vol. 16(1).
    12. David Moher & Alessandro Liberati & Jennifer Tetzlaff & Douglas G Altman & The PRISMA Group, 2009. "Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement," PLOS Medicine, Public Library of Science, vol. 6(7), pages 1-6, July.
    13. Kenneth F Schulz & Douglas G Altman & David Moher & for the CONSORT Group, 2010. "CONSORT 2010 Statement: Updated Guidelines for Reporting Parallel Group Randomised Trials," PLOS Medicine, Public Library of Science, vol. 7(3), pages 1-7, March.
    14. Mike Thelwall & Kayvan Kousha, 2016. "Are citations from clinical trials evidence of higher impact research? An analysis of ClinicalTrials.gov," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1341-1351, November.
    15. Franz Barjak & Xuemei Li & Mike Thelwall, 2007. "Which factors explain the Web impact of scientists' personal homepages?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(2), pages 200-211, January.
    16. Mike Thelwall, 2006. "Interpreting social science link analysis research: A theoretical framework," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(1), pages 60-68, January.
    17. José-Antonio Ontalba-Ruipérez & Enrique Orduna-Malea & Adolfo Alonso-Arroyo, 2016. "Identifying institutional relationships in a geographically distributed public health system using interlinking and co-authorship methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1167-1191, March.
    18. Martin Klein & Herbert Van de Sompel & Robert Sanderson & Harihar Shankar & Lyudmila Balakireva & Ke Zhou & Richard Tobin, 2014. "Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot," PLOS ONE, Public Library of Science, vol. 9(12), pages 1-39, December.
    19. Fatih Oguz & Wallace Koehler, 2016. "URL decay at year 20: A research note," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(2), pages 477-479, February.
    20. Mike Thelwall & Alesia Zuccala, 2008. "A university-centred European Union link analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(3), pages 407-420, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Enrique Orduña-Malea & Cristina I. Font-Julián & Jorge Serrano-Cobos, 2024. "Open access publications drive few visits from Google Search results to institutional repositories," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 7131-7152, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Enrique Orduña-Malea, 2021. "Dot-science top level domain: Academic websites or dumpsites?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3565-3591, April.
    2. Benedetto Lepori & Isidro F. Aguillo & Marco Seeber, 2014. "Size of web domains and interlinking behavior of higher education institutions in Europe," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 497-518, August.
    3. Patrick Kenekayoro & Kevan Buckley & Mike Thelwall, 2015. "Clustering research group website homepages," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2023-2039, March.
    4. Enrique Orduña-Malea & Cristina I. Font-Julián, 2022. "Are patents linked on Twitter? A case study of Google patents," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6339-6362, November.
    5. Simone Belli & Carlos Gonzalo-Penela, 2020. "Science, research, and innovation infospheres in Google results of the Ibero-American countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 635-653, May.
    6. Shalin Lee Wan Fei & Khatijah L Abdullah, 2015. "Effect of turning vs. supine position under phototherapy on neonates with hyperbilirubinemia: a systematic review," Journal of Clinical Nursing, John Wiley & Sons, vol. 24(5-6), pages 672-682, March.
    7. Savita Bakhshi & Alison E. While, 2013. "Health Professionals’ Alcohol-Related Professional Practices and the Relationship between Their Personal Alcohol Attitudes and Behavior and Professional Practices: A Systematic Review," IJERPH, MDPI, vol. 11(1), pages 1-31, December.
    8. Mike Thelwall, 2012. "Journal impact evaluation: a webometric perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 429-441, August.
    9. Mulhall, Peter & Taggart, Laurence & Coates, Vivien & McAloon, Toni & Hassiotis, Angela, 2018. "A systematic review of the methodological and practical challenges of undertaking randomised-controlled trials with cognitive disability populations," Social Science & Medicine, Elsevier, vol. 200(C), pages 114-128.
    10. Mike Thelwall, 2021. "Measuring Societal Impacts Of Research With Altmetrics? Common Problems And Mistakes," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1302-1314, December.
    11. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    12. Faith Donald & Kelley Kilpatrick & Kim Reid & Nancy Carter & Ruth Martin-Misener & Denise Bryant-Lukosius & Patricia Harbman & Sharon Kaasalainen & Deborah A. Marshall & Renee Charbonneau-Smith & Erin, 2014. "A Systematic Review of the Cost-Effectiveness of Nurse Practitioners and Clinical Nurse Specialists: What Is the Quality of the Evidence?," Nursing Research and Practice, Hindawi, vol. 2014, pages 1-28, September.
    13. Kartika Saraswati & Brittany J Maguire & Alistair R D McLean & Sauman Singh-Phulgenda & Roland C Ngu & Paul N Newton & Nicholas P J Day & Philippe J Guérin, 2021. "Systematic review of the scrub typhus treatment landscape: Assessing the feasibility of an individual participant-level data (IPD) platform," PLOS Neglected Tropical Diseases, Public Library of Science, vol. 15(10), pages 1-22, October.
    14. Muhammad Omar & Arif Mehmood & Gyu Sang Choi & Han Woo Park, 2017. "Global mapping of artificial intelligence in Google and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1269-1305, December.
    15. Ana Virgolino & Osvaldo Santos & Joana Costa & Mónica Fialho & Ivo Iavicoli & Tiina Santonen & Hanna Tolonen & Evangelia Samoli & Klea Katsouyanni & Georgios Baltatzis & Flavia Ruggieri & Annalisa Abb, 2021. "Challenges to Evidence Synthesis and Identification of Data Gaps in Human Biomonitoring," IJERPH, MDPI, vol. 18(6), pages 1-11, March.
    16. George Masterton & Erik J. Olsson & Staffan Angere, 2016. "Linking as voting: how the Condorcet jury theorem in political science is relevant to webometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 945-966, March.
    17. José-Antonio Ontalba-Ruipérez & Enrique Orduna-Malea & Adolfo Alonso-Arroyo, 2016. "Identifying institutional relationships in a geographically distributed public health system using interlinking and co-authorship methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1167-1191, March.
    18. Anaïs Besson & Alice Tarpin & Valentin Flaudias & Georges Brousse & Catherine Laporte & Amanda Benson & Valentin Navel & Jean-Baptiste Bouillon-Minois & Frédéric Dutheil, 2021. "Smoking Prevalence among Physicians: A Systematic Review and Meta-Analysis," IJERPH, MDPI, vol. 18(24), pages 1-58, December.
    19. Gianfranco Sanson & Ercole Vellone & Mari Kangasniemi & Rosaria Alvaro & Fabio D'Agostino, 2017. "Impact of nursing diagnoses on patient and organisational outcomes: a systematic literature review," Journal of Clinical Nursing, John Wiley & Sons, vol. 26(23-24), pages 3764-3783, December.
    20. Pei-Ying Kobres & Jean-Paul Chretien & Michael A Johansson & Jeffrey J Morgan & Pai-Yei Whung & Harshini Mukundan & Sara Y Del Valle & Brett M Forshey & Talia M Quandelacy & Matthew Biggerstaff & Ceci, 2019. "A systematic review and evaluation of Zika virus forecasting and prediction research during a public health emergency of international concern," PLOS Neglected Tropical Diseases, Public Library of Science, vol. 13(10), pages 1-21, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:128:y:2023:i:1:d:10.1007_s11192-022-04542-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.