IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0155060.html
   My bibliography  Save this article

How Criterion Scores Predict the Overall Impact Score and Funding Outcomes for National Institutes of Health Peer-Reviewed Applications

Author

Listed:
  • Matthew K Eblen
  • Robin M Wagner
  • Deepshikha RoyChowdhury
  • Katherine C Patel
  • Katrina Pearson

Abstract

Understanding the factors associated with successful funding outcomes of research project grant (R01) applications is critical for the biomedical research community. R01 applications are evaluated through the National Institutes of Health (NIH) peer review system, where peer reviewers are asked to evaluate and assign scores to five research criteria when assessing an application’s scientific and technical merit. This study examined the relationship of the five research criterion scores to the Overall Impact score and the likelihood of being funded for over 123,700 competing R01 applications for fiscal years 2010 through 2013. The relationships of other application and applicant characteristics, including demographics, to scoring and funding outcomes were studied as well. The analyses showed that the Approach and, to a lesser extent, the Significance criterion scores were the main predictors of an R01 application’s Overall Impact score and its likelihood of being funded. Applicants might consider these findings when submitting future R01 applications to NIH.

Suggested Citation

  • Matthew K Eblen & Robin M Wagner & Deepshikha RoyChowdhury & Katherine C Patel & Katrina Pearson, 2016. "How Criterion Scores Predict the Overall Impact Score and Funding Outcomes for National Institutes of Health Peer-Reviewed Applications," PLOS ONE, Public Library of Science, vol. 11(6), pages 1-17, June.
    • Handle: RePEc:plo:pone00:0155060
    DOI: 10.1371/journal.pone.0155060
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0155060
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0155060&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0155060?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Viner, Neil & Powell, Philip & Green, Rod, 2004. "Institutionalized biases in the award of research grants: a preliminary analysis revisiting the principle of accumulative advantage," Research Policy, Elsevier, vol. 33(3), pages 443-454, April.
    2. Michael R Martin & Andrea Kopstein & Joy M Janice, 2010. "An Analysis of Preliminary and Post-Discussion Priority Scores for Grant Applications Peer Reviewed by the Center for Scientific Review at the NIH," PLOS ONE, Public Library of Science, vol. 5(11), pages 1-6, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stephen A Gallo & Afton S Carpenter & Scott R Glisson, 2013. "Teleconference versus Face-to-Face Scientific Peer Review of Grant Application: Effects on Review Outcomes," PLOS ONE, Public Library of Science, vol. 8(8), pages 1-9, August.
    2. Qianshuo Liu & David Pérez-Castrillo & Inés Macho-Stadler & Albert Banal-Estañol, 2021. "Similar-to-me Effects in the Grant Application Process: Applicants, Panelists, and the Likelihood of Obtaining Funds," Working Papers 1289, Barcelona School of Economics.
    3. Simen G. Enger & Fulvio Castellacci, 2016. "Who gets Horizon 2020 research grants? Propensity to apply and probability to succeed in a two-step analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1611-1638, December.
    4. Day, Theodore Eugene, 2015. "The big consequences of small biases: A simulation of peer review," Research Policy, Elsevier, vol. 44(6), pages 1266-1270.
    5. Marco Cozzi, 2020. "Public Funding of Research and Grant Proposals in the Social Sciences: Empirical Evidence from Canada," Department Discussion Papers 1809, Department of Economics, University of Victoria.
    6. Banal-Estañol, Albert & Macho-Stadler, Inés & Pérez-Castrillo, David, 2019. "Evaluation in research funding agencies: Are structurally diverse teams biased against?," Research Policy, Elsevier, vol. 48(7), pages 1823-1840.
    7. Chiara Franzoni & Paula Stephan & Reinhilde Veugelers, 2022. "Funding Risky Research," Entrepreneurship and Innovation Policy and the Economy, University of Chicago Press, vol. 1(1), pages 103-133.
    8. Lawson, Cornelia & Salter, Ammon, 2023. "Exploring the effect of overlapping institutional applications on panel decision-making," Research Policy, Elsevier, vol. 52(9).
    9. David G Pina & Darko Hren & Ana Marušić, 2015. "Peer Review Evaluation Process of Marie Curie Actions under EU’s Seventh Framework Programme for Research," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-15, June.
    10. Simen G. Enger, 2017. "Closed clubs: Cumulative advantages and participation in Horizon 2020," Working Papers on Innovation Studies 20170703, Centre for Technology, Innovation and Culture, University of Oslo.
    11. Albert Banal-Estañol & Ines Macho-Stadler & David Pérez-Castrillo, 2016. "Key Success Drivers in Public Research Grants: Funding the Seeds of Radical Innovation in Academia?," CESifo Working Paper Series 5852, CESifo.
    12. Marjolijn N. Wijnen & Jorg J. M. Massen & Mariska E. Kret, 2021. "Gender bias in the allocation of student grants," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5477-5488, July.
    13. Margit Osterloh & Bruno S. Frey, 2010. "Academic rankings and research governance," IEW - Working Papers 482, Institute for Empirical Research in Economics - University of Zurich.
    14. Albert Banal-Estañol & Inés Macho-Stadler & David Pérez-Castillo, 2019. "Funding academic research: grant application, partnership, award, and output," Economics Working Papers 1658, Department of Economics and Business, Universitat Pompeu Fabra.
    15. O’Kane, Conor, 2018. "Technology transfer executives' backwards integration: An examination of interactions between university technology transfer executives and principal investigators," Technovation, Elsevier, vol. 76, pages 64-77.
    16. Margit Osterloh & Bruno S. Frey, 2009. "Research Governance in Academia: Are there Alternatives to Academic Rankings?," CREMA Working Paper Series 2009-17, Center for Research in Economics, Management and the Arts (CREMA).
    17. Lettice, Fiona & Smart, Palie & Baruch, Yehuda & Johnson, Mark, 2012. "Navigating the impact-innovation double hurdle: The case of a climate change research fund," Research Policy, Elsevier, vol. 41(6), pages 1048-1057.
    18. Kevin W. Boyack & Caleb Smith & Richard Klavans, 2018. "Toward predicting research proposal success," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 449-461, February.
    19. Benedetto Lepori & Valerio Veglio & Barbara Heller-Schuh & Thomas Scherngell & Michael Barber, 2015. "Participations to European Framework Programs of higher education institutions and their association with organizational characteristics," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 2149-2178, December.
    20. Grimpe, Christoph, 2012. "Extramural research grants and scientists’ funding strategies: Beggars cannot be choosers?," Research Policy, Elsevier, vol. 41(8), pages 1448-1460.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0155060. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.