IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v117y2018i1d10.1007_s11192-018-2852-1.html
   My bibliography  Save this article

Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups

Author

Listed:
  • Balázs Győrffy

    (Hungarian Academy of Sciences
    Semmelweis University)

  • Andrea Magda Nagy

    (University of Pannonia)

  • Péter Herman

    (Hungarian Academy of Sciences)

  • Ádám Török

    (University of Pannonia
    Budapest University of Technology and Economics)

Abstract

The Momentum program launched in 2009 provides funding of up to 1 million Euro to establish new, independent research groups at Hungarian academic institutions. Here, our aim was to determine factors associated with the scientific output of these research groups. Publication data were downloaded from the Hungarian Scientific Work Archive ( www.mtmt.hu ), impact factor data were obtained from Thomson Reuters (jcr.incites.thomsonreuters.com), and journal ranks were extracted from the Scimago Journal Rank database ( www.scimagojr.com ). Investigated input features for each grant holder include gender, degree, targeted category, international mobility, international grants, number of publications, total number of citations, H-index, best publications, impact factors in the last 2 years, and assessment scores provided by the experts. Evaluated performance indicators include cumulative impact factor, number of D1 publications, and number of first/last author D1 publications during the grant running time. Grant holders’ publication output increased by 23 and 52% for life sciences and material sciences researchers. Scientific performance was independent from gender, degree, international grants, category applied for, and citations received for the best pre-grant publication. Those with international mobility had significantly lower scientific output (yearly impact factor, number of D1 publications, number of first/last author publications). Scores received from grant review experts were independent from later publication activity. The strongest correlations were observed between scientific output and total number of citations, H-index, and impact factor in the last 2 years pre-grant. In summary, group leaders with a dynamic publication track record were able to attain the most additional momentum. Our results can help accelerate and improve future grant review processes.

Suggested Citation

  • Balázs Győrffy & Andrea Magda Nagy & Péter Herman & Ádám Török, 2018. "Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 409-426, October.
    • Handle: RePEc:spr:scient:v:117:y:2018:i:1:d:10.1007_s11192-018-2852-1
    DOI: 10.1007/s11192-018-2852-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-018-2852-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-018-2852-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Marianne Hörlesberger & Ivana Roche & Dominique Besagni & Thomas Scherngell & Claire François & Pascal Cuxac & Edgar Schiebel & Michel Zitt & Dirk Holste, 2013. "A concept for inferring ‘frontier research’ in grant proposals," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 129-148, November.
    2. Cristian Mejia & Yuya Kajikawa, 2018. "Using acknowledgement data to characterize funding organizations by the types of research sponsored: the case of robotics research," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 883-904, March.
    3. Katarína Cechlárová & Tamás Fleiner & Eva Potpinková, 2014. "Assigning evaluators to research grant applications: the case of Slovak Research and Development Agency," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 495-506, May.
    4. Pierre Azoulay & Joshua S. Graff Zivin & Gustavo Manso, 2011. "Incentives and creativity: evidence from the academic life sciences," RAND Journal of Economics, RAND Corporation, vol. 42(3), pages 527-554, September.
    5. Salil Gunashekar & Steven Wooding & Susan Guthrie, 2017. "How do NIHR peer review panels use bibliometric information to support their decisions?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1813-1835, September.
    6. Ulf Sandström & Martin Hällsten, 2008. "Persistent nepotism in peer-review," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 175-189, February.
    7. Erjia Yan & Chaojiang Wu & Min Song, 2018. "The funding factor: a cross-disciplinary examination of the association between research funding and citation impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 369-384, April.
    8. Niccolò Casnici & Francisco Grimaldo & Nigel Gilbert & Pierpaolo Dondio & Flaminio Squazzoni, 2017. "Assessing peer review by gauging the fate of rejected manuscripts: the case of the Journal of Artificial Societies and Social Simulation," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 533-546, October.
    9. Christina H. Drew & Kristianna G. Pettibone & Fallis Owen Finch & Douglas Giles & Paul Jordan, 2016. "Automated Research Impact Assessment: a new bibliometrics approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 987-1005, March.
    10. J. Rigby & K. Julian, 2014. "On the horns of a dilemma: does more funding for research lead to more research or a waste of resources that calls for optimization of researcher portfolios? An analysis using funding acknowledgement ," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1067-1075, November.
    11. Martin Reinhart, 2009. "Peer review of grant applications in biology and medicine. Reliability, fairness, and validity," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 789-809, December.
    12. Glenn D. Walters, 2016. "Adding authorship order to the quantity and quality dimensions of scholarly productivity: evidence from group- and individual-level analyses," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 769-785, February.
    13. Simen G. Enger & Fulvio Castellacci, 2016. "Who gets Horizon 2020 research grants? Propensity to apply and probability to succeed in a two-step analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1611-1638, December.
    14. Jianhua Hou & Xiucai Yang & Chaomei Chen, 2018. "Emerging trends and new developments in information science: a document co-citation analysis (2009–2016)," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 869-892, May.
    15. Guillermo Armando Ronda-Pupo & J. Sylvan Katz, 2018. "The power law relationship between citation impact and multi-authorship patterns in articles in Information Science & Library Science journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 919-932, March.
    16. Rodrigo Costas & Thomas Franssen, 2018. "Reflections around ‘the cautionary use’ of the h-index: response to Teixeira da Silva and Dobránszki," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1125-1130, May.
    17. Star X. Zhao & Wen Lou & Alice M. Tan & Shuang Yu, 2018. "Do funded papers attract more usage?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 153-168, April.
    18. John Rigby, 2009. "Comparing the scientific quality achieved by funding instruments for single grant holders and for collaborative networks within a research system: Some observations," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(1), pages 145-164, January.
    19. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia, 2018. "Competition between academic journals for scholars’ attention: the ‘Nature effect’ in scholarly communication," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1413-1432, June.
    20. Yves Gingras & Mahdi Khelfaoui, 2018. "Assessing the effect of the United States’ “citation advantage” on other countries’ scientific impact as measured in the Web of Science (WoS) database," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 517-532, February.
    21. Michael Thelwall, 2018. "Can Microsoft Academic be used for citation analysis of preprint archives? The case of the Social Science Research Network," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 913-928, May.
    22. Lutz Bornmann & Loet Leydesdorff, 2018. "Count highly-cited papers instead of papers with h citations: use normalized citation counts and compare “like with like”!," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1119-1123, May.
    23. Vladimir Batagelj & Anuška Ferligoj & Flaminio Squazzoni, 2017. "The emergence of a field: a network analysis of research on peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 503-532, October.
    24. José Luis Ortega, 2017. "Are peer-review activities related to reviewer bibliometric performance? A scientometric analysis of Publons," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 947-962, August.
    25. Thomas Heinze, 2008. "How to sponsor ground-breaking research: A comparison of funding schemes," Science and Public Policy, Oxford University Press, vol. 35(5), pages 302-318, June.
    26. Claudia Tania Picinin & Luiz Alberto Pilatti & João Luiz Kovaleski & Alexandre Reis Graeml & Bruno Pedroso, 2016. "Comparison of performance of researchers recipients of CNPq productivity grants in the field of Brazilian production engineering," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 855-870, November.
    27. J. Rigby & D. Cox & K. Julian, 2018. "Journal peer review: a bar or bridge? An analysis of a paper’s revision history and turnaround time, and the effect on citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1087-1105, March.
    28. Johan Bollen & David Crandall & Damion Junk & Ying Ding & Katy Börner, 2017. "An efficient system to fund science: from proposal review to peer-to-peer distributions," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 521-528, January.
    29. Xingchen Li & Qiang Wu & Yuanyuan Liu, 2017. "A quantitative analysis of researcher citation personal display considering disciplinary differences and influence factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1093-1112, November.
    30. Sotaro Shibayama, 2011. "Distribution of academic research funds: a case of Japanese national research grant," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 43-60, July.
    31. Mu-Hsuan Huang & Mei-Jhen Huang, 2018. "An analysis of global research funding from subject field and funding agencies perspectives in the G9 countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 833-847, May.
    32. João Carlos Nabout & Fabrício Barreto Teresa & Karine Borges Machado & Vitor Hugo Mendonça Prado & Luis Mauricio Bini & José Alexandre Felizola Diniz-Filho, 2018. "Do traditional scientometric indicators predict social media activity on scientific knowledge? An analysis of the ecological literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1007-1015, May.
    33. Jens Jirschitzka & Aileen Oeberst & Richard Göllner & Ulrike Cress, 2017. "Inter-rater reliability and validity of peer reviews in an interdisciplinary field," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1059-1092, November.
    34. Marek Kwiek, 2018. "High research productivity in vertically undifferentiated higher education systems: Who are the top performers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 415-462, April.
    35. Lutz Bornmann & Klaus Wohlrabe & Felix Moya Anegon, 2017. "Calculating the excellence shift: How efficiently do institutions produce highly cited papers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1859-1864, September.
    36. Charles W. Fox, 2017. "Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 465-477, October.
    37. Peng Bao & Chengxiang Zhai, 2017. "Dynamic credit allocation in scientific literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 595-606, July.
    38. C. Sean Burns & Charles W. Fox, 2017. "Language and socioeconomics predict geographic variation in peer review outcomes at an ecology journal," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1113-1127, November.
    39. Hui Fang, 2011. "Peer review and over-competitive research funding fostering mainstream opinion to monopoly," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 293-301, May.
    40. Michail Kovanis & Ludovic Trinquart & Philippe Ravaud & Raphaël Porcher, 2017. "Evaluating alternative systems of peer review: a large-scale agent-based modelling approach to scientific publication," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 651-671, October.
    41. Aliakbar Akbaritabar & Niccolò Casnici & Flaminio Squazzoni, 2018. "The conundrum of research productivity: a study on sociologists in Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 859-882, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Junwan Liu & Rui Wang & Shuo Xu, 2021. "What academic mobility configurations contribute to high performance: an fsQCA analysis of CSC-funded visiting scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1079-1100, February.
    2. Balázs Győrffy & Gyöngyi Csuka & Péter Herman & Ádám Török, 2020. "Is there a golden age in publication activity?—an analysis of age-related scholarly performance across all scientific disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1081-1097, August.
    3. Török, Ádám & Nagy, Andrea Magda, 2021. "A "nagy ugrás" a felsőoktatásban - Kína a nemzetközi verseny élmezőnyében? [The big leap" in higher education - Has China moved into the lead?]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(4), pages 375-398.
    4. Győrffy, Balázs & Herman, Péter & Szabó, István, 2020. "Research funding: past performance is a stronger predictor of future scientific output than reviewer scores," Journal of Informetrics, Elsevier, vol. 14(3).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Weishu Liu & Li Tang & Guangyuan Hu, 2020. "Funding information in Web of Science: an updated overview," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1509-1524, March.
    2. Qianqian Jin & Hongshu Chen & Ximeng Wang & Tingting Ma & Fei Xiong, 2022. "Exploring funding patterns with word embedding-enhanced organization–topic networks: a case study on big data," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5415-5440, September.
    3. Kok, Holmer & Faems, Dries & de Faria, Pedro, 2022. "Pork Barrel or Barrel of Gold? Examining the performance implications of earmarking in public R&D grants," Research Policy, Elsevier, vol. 51(7).
    4. Kevin W. Boyack & Caleb Smith & Richard Klavans, 2018. "Toward predicting research proposal success," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 449-461, February.
    5. Akbaritabar, Aliakbar & Stephen, Dimity & Squazzoni, Flaminio, 2022. "A study of referencing changes in preprint-publication pairs across multiple fields," Journal of Informetrics, Elsevier, vol. 16(2).
    6. Melika Mosleh & Saeed Roshani & Mario Coccia, 2022. "Scientific laws of research funding to support citations and diffusion of knowledge in life science," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1931-1951, April.
    7. Manoj Kumar Verma & Daud Khan & Mayank Yuvaraj, 2023. "Scientometric assessment of funded scientometrics and bibliometrics research (2011–2021)," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4305-4320, August.
    8. Saeed Roshani & Mohammad-Reza Bagherylooieh & Melika Mosleh & Mario Coccia, 2021. "What is the relationship between research funding and citation-based performance? A comparative analysis between critical disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7859-7874, September.
    9. Alexandre Rodrigues Oliveira & Carlos Fernando Mello, 2016. "Importance and susceptibility of scientific productivity indicators: two sides of the same coin," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 697-722, November.
    10. Gerald Schweiger & Adrian Barnett & Peter van den Besselaar & Lutz Bornmann & Andreas De Block & John P. A. Ioannidis & Ulf Sandstrom & Stijn Conix, 2024. "The Costs of Competition in Distributing Scarce Research Funds," Papers 2403.16934, arXiv.org.
    11. Corsini, Alberto & Pezzoni, Michele, 2023. "Does grant funding foster research impact? Evidence from France," Journal of Informetrics, Elsevier, vol. 17(4).
    12. Youtie, Jan & Rogers, Juan & Heinze, Thomas & Shapira, Philip & Tang, Li, 2013. "Career-based influences on scientific recognition in the United States and Europe: Longitudinal evidence from curriculum vitae data," Research Policy, Elsevier, vol. 42(8), pages 1341-1355.
    13. Seeber, Marco & Alon, Ilan & Pina, David G. & Piro, Fredrik Niclas & Seeber, Michele, 2022. "Predictors of applying for and winning an ERC Proof-of-Concept grant: An automated machine learning model," Technological Forecasting and Social Change, Elsevier, vol. 184(C).
    14. Jinyang Dong & Jiamou Liu & Tiezhong Liu, 2021. "The impact of top scientists on the community development of basic research directed by government funding: evidence from program 973 in China," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(10), pages 8561-8579, October.
    15. Katarína Cechlárová & Tamás Fleiner & Eva Potpinková, 2014. "Assigning evaluators to research grant applications: the case of Slovak Research and Development Agency," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 495-506, May.
    16. Bianchi, Federico & Grimaldo, Francisco & Squazzoni, Flaminio, 2019. "The F3-index. Valuing reviewers for scholarly journals," Journal of Informetrics, Elsevier, vol. 13(1), pages 78-86.
    17. Luukkonen, Terttu, 2014. "Universities, Funding Systems, and the Renewal of the Industrial Knowledge Base – UNI Project Findings," ETLA Reports 33, The Research Institute of the Finnish Economy.
    18. Li, Kai & Yan, Erjia, 2019. "Are NIH-funded publications fulfilling the proposed research? An examination of concept-matchedness between NIH research grants and their supported publications," Journal of Informetrics, Elsevier, vol. 13(1), pages 226-237.
    19. Thomas Feliciani & Junwen Luo & Lai Ma & Pablo Lucas & Flaminio Squazzoni & Ana Marušić & Kalpana Shankar, 2019. "A scoping review of simulation models of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 555-594, October.
    20. Zhu, Wanying & Jin, Ching & Ma, Yifang & Xu, Cong, 2023. "Earlier recognition of scientific excellence enhances future achievements and promotes persistence," Journal of Informetrics, Elsevier, vol. 17(2).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:117:y:2018:i:1:d:10.1007_s11192-018-2852-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.