IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v94y2013i1d10.1007_s11192-012-0779-5.html
   My bibliography  Save this article

Looking for the impact of peer review: does count of funding acknowledgements really predict research impact?

Author

Listed:
  • John Rigby

    (University of Manchester)

Abstract

A small number of studies have sought to establish that research papers with more funding acknowledgements achieve higher impact and have claimed that such a link exists because research supported by more funding bodies undergoes more peer review. In this paper, a test of this link is made using recently available data from the Web of Science, a source of bibliographic data that now includes funding acknowledgements. The analysis uses 3,596 papers from a single year, 2009, and a single journal, the Journal of Biological Chemistry. Analysis of this data using OLS regression and two ranks tests reveals the link between count of funding acknowledgements and high impact papers to be statistically significant, but weak. It is concluded that count of funding acknowledgements should not be considered a reliable indicator of research impact at this level. Relatedly, indicators based on assumptions that may hold true at one level of analysis may not be appropriate at other levels.

Suggested Citation

  • John Rigby, 2013. "Looking for the impact of peer review: does count of funding acknowledgements really predict research impact?," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(1), pages 57-73, January.
    • Handle: RePEc:spr:scient:v:94:y:2013:i:1:d:10.1007_s11192-012-0779-5
    DOI: 10.1007/s11192-012-0779-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-012-0779-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-012-0779-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Stefan Hornbostel, 2001. "Third party funding of German universities. An indicator of research activity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 50(3), pages 523-537, March.
    2. John Rigby, 2011. "Systematic grant and funding body acknowledgement data for publications: new dimensions and new controversies for research policy and evaluation," Research Evaluation, Oxford University Press, vol. 20(5), pages 365-375, December.
    3. Grant Lewison & Valentina Markusova, 2010. "The evaluation of Russian cancer research," Research Evaluation, Oxford University Press, vol. 19(2), pages 129-144, June.
    4. Ulf Sandström, 2009. "Research quality and diversity of funding: A model for relating research money to output of research," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(2), pages 341-349, May.
    5. Francis Narin & Gabriel Pinski & Helen Hofer Gee, 1976. "Structure of the Biomedical Literature," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 27(1), pages 25-45, January.
    6. T N van Leeuwen & L J van der Wurff & A F J van Raan, 2001. "The use of combined bibliometric methods in research funding policy," Research Evaluation, Oxford University Press, vol. 10(3), pages 195-201, December.
    7. Linda Butler, 2001. "Revisiting bibliometric issues using new empirical data," Research Evaluation, Oxford University Press, vol. 10(1), pages 59-65, April.
    8. Heinze, Thomas & Shapira, Philip & Rogers, Juan D. & Senker, Jacqueline M., 2009. "Organizational and institutional influences on creativity in scientific research," Research Policy, Elsevier, vol. 38(4), pages 610-623, May.
    9. Nick Haslam & Lauren Ban & Leah Kaufmann & Stephen Loughnan & Kim Peters & Jennifer Whelan & Sam Wilson, 2008. "What makes an article influential? Predicting impact in social and personality psychology," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(1), pages 169-185, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Belén Álvarez-Bornstein & Adrián A. Díaz-Faes & María Bordons, 2019. "What characterises funded biomedical research? Evidence from a basic and a clinical domain," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 805-825, May.
    2. Jue Wang & Philip Shapira, 2011. "Funding acknowledgement analysis: an enhanced tool to investigate research sponsorship impacts: the case of nanotechnology," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 563-586, June.
    3. Corsini, Alberto & Pezzoni, Michele, 2023. "Does grant funding foster research impact? Evidence from France," Journal of Informetrics, Elsevier, vol. 17(4).
    4. Nicola Grassano & Daniele Rotolo & Joshua Hutton & Frédérique Lang & Michael M. Hopkins, 2017. "Funding Data from Publication Acknowledgments: Coverage, Uses, and Limitations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(4), pages 999-1017, April.
    5. Álvarez-Bornstein, Belén & Bordons, María, 2021. "Is funding related to higher research impact? Exploring its relationship and the mediating role of collaboration in several disciplines," Journal of Informetrics, Elsevier, vol. 15(1).
    6. Lili Miao & Vincent Larivi`ere & Feifei Wang & Yong-Yeol Ahn & Cassidy R. Sugimoto, 2023. "Cooperation and interdependence in global science funding," Papers 2308.08630, arXiv.org, revised Feb 2024.
    7. Tomas Hellström, 2018. "Centres of Excellence and Capacity Building: from Strategy to Impact," Science and Public Policy, Oxford University Press, vol. 45(4), pages 543-552.
    8. Cristian Mejia & Yuya Kajikawa, 2018. "Using acknowledgement data to characterize funding organizations by the types of research sponsored: the case of robotics research," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 883-904, March.
    9. Shibayama, Sotaro & Baba, Yasunori & Walsh, John P., 2015. "Organizational design of University laboratories: Task allocation and lab performance in Japanese bioscience laboratories," Research Policy, Elsevier, vol. 44(3), pages 610-622.
    10. Xianwen Wang & Di Liu & Kun Ding & Xinran Wang, 2012. "Science funding and research output: a study on 10 countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 591-599, May.
    11. Alberto Corsini & Michele Pezzoni, 2022. "Does grant funding foster research impact? Evidence from France," SciencePo Working papers Main hal-03912647, HAL.
    12. Adèle Paul-Hus & Nadine Desrochers & Rodrigo Costas, 2016. "Characterization, description, and considerations for the use of funding acknowledgement data in Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 167-182, July.
    13. Alberto Corsini & Michele Pezzoni, 2022. "Does grant funding foster research impact? Evidence from France," Working Papers hal-03912647, HAL.
    14. Michael Fritsch & Viktor Slavtchev, 2007. "What determines the efficiency of regional innovation systems?," Jena Economics Research Papers 2007-006, Friedrich-Schiller-University Jena.
    15. M. Teresa Antonio-García & Irene López-Navarro & Jesús Rey-Rocha, 2014. "Determinants of success for biomedical researchers: a perception-based study in a health science research environment," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1747-1779, December.
    16. Gao, Qiang & Liang, Zhentao & Wang, Ping & Hou, Jingrui & Chen, Xiuxiu & Liu, Manman, 2021. "Potential index: Revealing the future impact of research topics based on current knowledge networks," Journal of Informetrics, Elsevier, vol. 15(3).
    17. Albert Banal-Estañol & Qianshuo Liu & Inés Macho-Stadler & David Pérez-Castrillo, 2021. "Similar-to-me Effects in the Grant Application Process: Applicants, Panelists, and the Likelihood of Obtaining Funds," Working Papers 1289, Barcelona School of Economics.
    18. Fiorenzo Franceschini & Maurizio Galetto & Domenico Maisano & Luca Mastrogiacomo, 2012. "The success-index: an alternative approach to the h-index for evaluating an individual’s research output," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 621-641, September.
    19. Elvira Uyarra & Jens Sörvik & Inger Midtkandal, 2014. "Inter-regional Collaboration in Research and Innovation Strategies for Smart Specialisation (RIS3). S3 Working Paper Series no 6/2014," JRC Research Reports JRC91963, Joint Research Centre.
    20. Boyack, Kevin W. & Patek, Michael & Ungar, Lyle H. & Yoon, Patrick & Klavans, Richard, 2014. "Classification of individual articles from all of science by research level," Journal of Informetrics, Elsevier, vol. 8(1), pages 1-12.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:94:y:2013:i:1:d:10.1007_s11192-012-0779-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.