IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v109y2016i2d10.1007_s11192-016-2047-6.html
   My bibliography  Save this article

Importance and susceptibility of scientific productivity indicators: two sides of the same coin

Author

Listed:
  • Alexandre Rodrigues Oliveira

    (Federal University of Santa Maria (UFSM))

  • Carlos Fernando Mello

    (Federal University of Santa Maria (UFSM))

Abstract

We investigated whether applicants or recipients of research productivity fellowships of the main research financing agency in Brazil (CNPq) would consider the most “important products and indicators” of scientific/academic activity those also considered the least susceptible. We hypothesized that perception of susceptibility and importance of productivity indicators would vary according to the fellowship level of the grantees. Seven hundred and two scientists, being 79 non-grantees and 623 recipients of research productivity fellowships in the area of biosciences participated in the study. The scientists were requested to score the importance of a series of indicators (i.e., total number of published articles, number of articles as first author, number of articles as last/corresponding author, H-index, books and others, totalizing 39 variables) using a Likert scale. After completing the evaluation of the symbolic importance of all indicators, the scientists scored the “susceptibility” of the same indicators. The most important products and indicators of productivity were also those considered the least susceptible. Local, national and international prizes, publications or grants were increasingly perceived as more important and less susceptible. Moreover, the symbolic magnitude of susceptibility and importance of the elements (indicators) of the curriculum varied according to the productivity fellowship level of the grantee and gender. Despite the observed differences, a consensus of the most important and least susceptible products and indicators could be established. Ultimate individual responsibility and international projection are common characteristics of the most important and least susceptible indicators of scientific productivity.

Suggested Citation

  • Alexandre Rodrigues Oliveira & Carlos Fernando Mello, 2016. "Importance and susceptibility of scientific productivity indicators: two sides of the same coin," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 697-722, November.
    • Handle: RePEc:spr:scient:v:109:y:2016:i:2:d:10.1007_s11192-016-2047-6
    DOI: 10.1007/s11192-016-2047-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-016-2047-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-016-2047-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Carillo, Maria Rosaria & Papagni, Erasmo & Sapio, Alessandro, 2013. "Do collaborations enhance the high-quality output of scientific institutions? Evidence from the Italian Research Assessment Exercise," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 47(C), pages 25-36.
    2. Mushin Lee & Kiyong Om & Joon Koh, 2000. "The Bias of Sighted Reviewers in Research Proposal Evaluation: A Comparative Analysis of Blind and Open Review in Korea," Scientometrics, Springer;Akadémiai Kiadó, vol. 48(1), pages 99-116, June.
    3. Rodrigo Costas & María Bordons, 2011. "Do age and professional rank influence the order of authorship in scientific publications? Some evidence from a micro-level perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 145-161, July.
    4. Christoph Bartneck & Servaas Kokkelmans, 2011. "Detecting h-index manipulation through self-citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 85-98, April.
    5. Ulf Sandström & Martin Hällsten, 2008. "Persistent nepotism in peer-review," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 175-189, February.
    6. Chaoqun Ni & Debora Shaw & Sean M. Lind & Ying Ding, 2013. "Journal impact and proximity: An assessment using bibliographic features," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(4), pages 802-817, April.
    7. Wolfgang Glänzel & Jacqueline Leta & Bart Thijs, 2006. "Science in Brazil. Part 1: A macro-level comparative study," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(1), pages 67-86, April.
    8. Fedderke, J.W. & Goldschmidt, M., 2015. "Does massive funding support of researchers work?: Evaluating the impact of the South African research chair funding initiative," Research Policy, Elsevier, vol. 44(2), pages 467-482.
    9. Wolfgang Glänzel & Jacqueline Leta & Bart Thijs, 2006. "Science in Brazil. Part 1: A macro-level comparative study," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(1), pages 67-86, April.
    10. Glenn D. Walters, 2016. "Adding authorship order to the quantity and quality dimensions of scholarly productivity: evidence from group- and individual-level analyses," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 769-785, February.
    11. Daniel Henrique Roos & Luciana Calabró & Sandra Lopes Jesus & Diogo Onofre Souza & Nilda Vargas Barbosa & João Batista Teixeira Rocha, 2014. "Brazilian scientific production in areas of biological sciences: a comparative study on the modalities of full doctorate in Brazil or abroad," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 415-427, January.
    12. Clara Eugenia Garcia & Luis Sanz-Menéndez, 2005. "Competition for funding as an indicator of research competitiveness," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(3), pages 271-300, August.
    13. Heather Piwowar, 2013. "Value all research products," Nature, Nature, vol. 493(7431), pages 159-159, January.
    14. Chaoqun Ni & Debora Shaw & Sean M. Lind & Ying Ding, 2013. "Journal impact and proximity: An assessment using bibliographic features," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(4), pages 802-817, April.
    15. Lutz Bornmann & Hans-Dieter Daniel, 2006. "Selecting scientific excellence through committee peer review - A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicants," Scientometrics, Springer;Akadémiai Kiadó, vol. 68(3), pages 427-440, September.
    16. Martin Reinhart, 2009. "Peer review of grant applications in biology and medicine. Reliability, fairness, and validity," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 789-809, December.
    17. Tianzhu Tao & Lulong Bo & Fei Wang & Jinbao Li & Xiaoming Deng, 2012. "Equal contributions and credit given to authors in anesthesiology journals during a 10-year period," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 1005-1010, June.
    18. L. Erik Clavería & Eliseo Guallar & Jordi Camí & José Conde & Roberto Pastor & José R. Ricoy & Eduardo Rodríguez-Farré & Fernando Ruiz-Palomo & Emilio Muñoz, 2000. "Does Peer Review Predict the Performance of Research Projects in Health Sciences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 47(1), pages 11-23, January.
    19. Lutz Bornmann & Hans-Dieter Daniel, 2005. "Selection of research fellowship recipients by committee peer review. Reliability, fairness and predictive validity of Board of Trustees' decisions," Scientometrics, Springer;Akadémiai Kiadó, vol. 63(2), pages 297-320, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Soleidy Rivero Amador & Maidelyn Díaz Pérez & María José López-Huertas & Reinaldo Javier Rodríguez Font, 2018. "Indicator system for managing science, technology and innovation in universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1575-1587, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Balázs Győrffy & Andrea Magda Nagy & Péter Herman & Ádám Török, 2018. "Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 409-426, October.
    2. Kevin W. Boyack & Caleb Smith & Richard Klavans, 2018. "Toward predicting research proposal success," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 449-461, February.
    3. Jacques Wainer & Eduardo C. Xavier & Fabio Bezerra, 2009. "Scientific production in Computer Science: A comparative study of Brazil and other countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(2), pages 535-547, November.
    4. Júlio Cesar Rodrigues Pereira & Juliana Parreira Vasconcellos & Lucilla Furusawa & Augusto De Moura Barbati, 2007. "Who’s who and what’s what in Brazilian Public Health Sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 73(1), pages 37-52, October.
    5. Mauro Vitor Mendlowicz & Evandro Silva Freire Coutinho & Jerson Laks & Leonardo Franklin Fontenelle & Alexandre Martins Valença & William Berger & Ivan Figueira & Gláucia Azambuja Aguiar, 2011. "Is there a ‘gender gap’ in authorship of the main Brazilian psychiatric journals at the beginning of the 21st century?," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 27-37, January.
    6. Carla Mara Hilário & Maria Cláudia Cabrini Grácio, 2017. "Scientific collaboration in Brazilian researches: a comparative study in the information science, mathematics and dentistry fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 929-950, November.
    7. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    8. Ping Zhou & Wolfgang Glänzel, 2010. "In-depth analysis on China’s international cooperation in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 597-612, March.
    9. Tóth, Tamás & Demeter, Márton & Csuhai, Sándor & Major, Zsolt Balázs, 2024. "When career-boosting is on the line: Equity and inequality in grant evaluation, productivity, and the educational backgrounds of Marie Skłodowska-Curie Actions individual fellows in social sciences an," Journal of Informetrics, Elsevier, vol. 18(2).
    10. Daniel Fink & Youngsun Kwon & Jae Jeung Rho & Minho So, 2014. "S&T knowledge production from 2000 to 2009 in two periphery countries: Brazil and South Korea," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(1), pages 37-54, April.
    11. Primož Južnič & Stojan Pečlin & Matjaž Žaucer & Tilen Mandelj & Miro Pušnik & Franci Demšar, 2010. "Scientometric indicators: peer-review, bibliometric methods and conflict of interests," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(2), pages 429-441, November.
    12. Harlley Lima & Thiago H. P. Silva & Mirella M. Moro & Rodrygo L. T. Santos & Wagner Meira & Alberto H. F. Laender, 2015. "Assessing the profile of top Brazilian computer science researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 879-896, June.
    13. Bornmann, Lutz & Mutz, Rüdiger & Daniel, Hans-Dieter, 2008. "Latent Markov modeling applied to grant peer review," Journal of Informetrics, Elsevier, vol. 2(3), pages 217-228.
    14. Zaida Chinchilla-Rodríguez & Grisel Zacca-González & Benjamín Vargas-Quesada & Félix Moya-Anegón, 2015. "Latin American scientific output in Public Health: combined analysis using bibliometric, socioeconomic and health indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 609-628, January.
    15. Azzurra Ragone & Katsiaryna Mirylenka & Fabio Casati & Maurizio Marchese, 2013. "On peer review in computer science: analysis of its effectiveness and suggestions for improvement," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 317-356, November.
    16. Denis Arruda & Fábio Bezerra & Vânia Almeida Neris & Patricia Rocha De Toro & Jacques Wainera, 2009. "Brazilian computer science research: Gender and regional distributions," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(3), pages 651-665, June.
    17. Timur Narbaev & Diana Amirbekova, 2021. "Research Productivity in Emerging Economies: Empirical Evidence from Kazakhstan," Publications, MDPI, vol. 9(4), pages 1-19, November.
    18. Croft, William L. & Sack, Jörg-Rüdiger, 2022. "Predicting the citation count and CiteScore of journals one year in advance," Journal of Informetrics, Elsevier, vol. 16(4).
    19. Wolfgang Glänzel & Koenraad Debackere & Martin Meyer, 2008. "‘Triad’ or ‘tetrad’? On global changes in a dynamic world," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(1), pages 71-88, January.
    20. Vieira, Elizabeth S. & Cabral, José A.S. & Gomes, José A.N.F., 2014. "How good is a model based on bibliometric indicators in predicting the final decisions made by peers?," Journal of Informetrics, Elsevier, vol. 8(2), pages 390-405.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:109:y:2016:i:2:d:10.1007_s11192-016-2047-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.