IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v127y2022i10d10.1007_s11192-022-04490-8.html
   My bibliography  Save this article

Use of bibliometrics for research evaluation in emerging markets economies: a review and discussion of bibliometric indicators

Author

Listed:
  • Hamdi A. Al-Jamimi

    (Research Institute, King Fahd University of Petroleum & Minerals)

  • Galal M. BinMakhashen

    (Research Institute, King Fahd University of Petroleum & Minerals)

  • Lutz Bornmann

    (Max Planck Institute for Solid State Research
    Administrative Headquarters of the Max Planck Society)

Abstract

The governments of most ‘emerging market’ (EM) countries (e.g., India, Mexico, and Brazil) have invested significantly in scientific research over the past decades. These investments are reflected in the countries’ research volume, impact, and international collaboration activities. The EMs’ research support justification delivers the need to evaluate their research performance at multi-levels (country, institution, and scholar), e.g., by using indicators of research application inclinations. Bibliometric studies are increasingly requested by research and development (R&D) policymakers, funding agencies, and other relevant actors in these countries. The results of the studies are used to develop research directions, set research policies, and satisfy management needs (at universities). In this paper, we review the contributions of previous (bibliometric) studies to measuring research performance in EM countries. We identified 132 indicators used in the studies and classified them into three categories: productivity, impact, and collaboration. Some indicators have been more frequently used than others for measuring the performance of EM countries. These indicators might be the most important ones. For example, research leadership indicators (focusing on corresponding authors of publications) might be more important than other indicators measuring the research productivity of EM countries since they highlight the ‘actual’ contribution of national scientists. Collaboration indicators measuring national/international collaborative publications are also especially relevant for EM countries; their results can be used to sharpen expertise and exchange of professional knowledge. The overview presented in this paper can be used for the planning and realization of future studies on research in EM countries.

Suggested Citation

  • Hamdi A. Al-Jamimi & Galal M. BinMakhashen & Lutz Bornmann, 2022. "Use of bibliometrics for research evaluation in emerging markets economies: a review and discussion of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5879-5930, October.
  • Handle: RePEc:spr:scient:v:127:y:2022:i:10:d:10.1007_s11192-022-04490-8
    DOI: 10.1007/s11192-022-04490-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-022-04490-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-022-04490-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jun-Ying Fu & Xu Zhang & Yun-Hua Zhao & Mu-Hsuan Huang & Dar-Zen Chen, 2011. "Bibliometric analysis of complementary and alternative medicine research over three decades," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 617-626, August.
    2. Lutz Bornmann, 2014. "How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature," Research Evaluation, Oxford University Press, vol. 23(2), pages 166-173.
    3. Gangan Prathap, 2011. "The Energy–Exergy–Entropy (or EEE) sequences in bibliometric assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 515-524, June.
    4. Muhammad Farooq & Muhammad Asim & Muhammad Imran & Shahid Imran & Jameel Ahmad & Muhammad Rizwan Younis, 2018. "Mapping past, current and future energy research trend in Pakistan: a scientometric assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1733-1753, December.
    5. V. A. Markusova & A. N. Libkind & A. E. Varshavsly & C. N. M. Jansz, 2012. "Research performance and collaboration in the Novosibirsk region," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 513-526, May.
    6. Adams, Jonathan, 2018. "Information and misinformation in bibliometric time-trend analysis," Journal of Informetrics, Elsevier, vol. 12(4), pages 1063-1071.
    7. Hamid Darvish & Yaşar Tonta, 2016. "Diffusion of nanotechnology knowledge in Turkey and its network structure," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 569-592, May.
    8. Elio Atenógenes Villaseñor & Ricardo Arencibia-Jorge & Humberto Carrillo-Calvet, 2017. "Multiparametric characterization of scientometric performance profiles assisted by neural networks: a study of Mexican higher education institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 77-104, January.
    9. Har Kaur & B. M. Gupta, 2010. "Mapping of dental science research in India: a scientometric analysis of India’s research output, 1999–2008," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 361-376, October.
    10. Veronica Perez-Cabezas & Carmen Ruiz-Molinero & Ines Carmona-Barrientos & Enrique Herrera-Viedma & Manuel J. Cobo & Jose A. Moral-Munoz, 2018. "Highly cited papers in rheumatology: identification and conceptual analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 555-568, July.
    11. Liang Zhang & Sisi Li & Hugo A. Loáiciga & Yanhua Zhuang & Yun Du, 2015. "Opportunities and challenges of interbasin water transfers: a literature review with bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(1), pages 279-294, October.
    12. Zaida Chinchilla-Rodríguez & Grisel Zacca-González & Benjamín Vargas-Quesada & Félix Moya-Anegón, 2015. "Latin American scientific output in Public Health: combined analysis using bibliometric, socioeconomic and health indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 609-628, January.
    13. Abdulkader A. Murad & Dimitar T. Tomov, 2012. "Institutionalization and internationalization of research on the applications of the geographical information systems in health planning," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(1), pages 143-158, April.
    14. Maria Karaulova & Abdullah Gök & Oliver Shackleton & Philip Shapira, 2016. "Science system path-dependencies and their influences: nanotechnology research in Russia," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 645-670, May.
    15. P. Nishy & Yatish Panwar & Suresh Prasad & G. K. Mandal & Gangan Prathap, 2012. "An impact-citations-exergy (iCX) trajectory analysis of leading research institutions in India," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(1), pages 245-251, April.
    16. Jing-Li Fan & Shuo Shen & Jian-Da Wang & Shi-Jie Wei & Xian Zhang & Ping Zhong & Hang Wang, 2020. "Scientific and technological power and international cooperation in the field of natural hazards: a bibliometric analysis," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 102(3), pages 807-827, July.
    17. Hsuan-I Liu & Bi-Chun Chang & Kuan-Chia Chen, 2012. "Collaboration patterns of Taiwanese scientific publications in various research areas," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(1), pages 145-155, July.
    18. Liu, Weishu & Hu, Guangyuan & Tang, Li & Wang, Yuandi, 2015. "China's global growth in social science research: Uncovering evidence from bibliometric analyses of SSCI publications (1978–2013)," Journal of Informetrics, Elsevier, vol. 9(3), pages 555-569.
    19. Jacqueline Leta & Wolfgang Glänzel & Bart Thijs, 2006. "Science in Brazil. Part 2: Sectoral and institutional research profiles," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(1), pages 87-105, April.
    20. Muzammil Tahira & Rose Alinda Alias & Aryati Bakri, 2013. "Scientometric assessment of engineering in Malaysians universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(3), pages 865-879, September.
    21. Qiang Yao & Peng-Hui Lyu & Lian-Ping Yang & Lan Yao & Zhi-Yong Liu, 2014. "Current performance and future trends in health care sciences and services research," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 751-779, October.
    22. Gangan Prathap, 2010. "The 100 most prolific economists using the p-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 167-172, July.
    23. Gustavo Cattelan Nobre & Elaine Tavares, 2017. "Scientific literature analysis on big data and internet of things applications on circular economy: a bibliometric study," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 463-492, April.
    24. I. Jarić & J. Gessner, 2012. "Analysis of publications on sturgeon research between 1996 and 2010," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 715-735, February.
    25. Robert J. W. Tijssen & Jos Winnink, 2016. "Twenty-first century macro-trends in the institutional fabric of science: bibliometric monitoring and analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2181-2194, December.
    26. Weishu Liu & Mengdi Gu & Guangyuan Hu & Chao Li & Huchang Liao & Li Tang & Philip Shapira, 2014. "Profile of developments in biomass-based bioenergy research: a 20-year perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 507-521, May.
    27. Xiao Han T Zeng & Jordi Duch & Marta Sales-Pardo & João A G Moreira & Filippo Radicchi & Haroldo V Ribeiro & Teresa K Woodruff & Luís A Nunes Amaral, 2016. "Differences in Collaboration Patterns across Discipline, Career Stage, and Gender," PLOS Biology, Public Library of Science, vol. 14(11), pages 1-19, November.
    28. Pardeep Sud & Mike Thelwall, 2014. "Evaluating altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1131-1143, February.
    29. Shakil Ahmad & Yasir Javed & Shabir Hussain Khahro & Arslan Shahid, 2020. "Research Contribution of the Oldest Seat of Higher Learning in Pakistan: A Bibliometric Analysis of University of the Punjab," Publications, MDPI, vol. 8(3), pages 1-15, September.
    30. Bronwyn H. Hall & Albert N. Link & John T. Scott, 2003. "Universities as Research Partners," The Review of Economics and Statistics, MIT Press, vol. 85(2), pages 485-491, May.
    31. Robert J. W. Tijssen & Martijn S. Visser & Thed N. van Leeuwen, 2002. "Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference?," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(3), pages 381-397, July.
    32. Köksal Şahin & Gökçe Candan, 2018. "Scientific productivity and cooperation in Turkic world: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1199-1229, June.
    33. Yaşar Tonta, 2000. "Contribution of Turkish Researchers to the World's Biomedical Literature (1988–1997)," Scientometrics, Springer;Akadémiai Kiadó, vol. 48(1), pages 71-84, June.
    34. Haijun Wang & Minyan Liu & Song Hong & Yanhua Zhuang, 2013. "A historical review and bibliometric analysis of GPS research from 1991–2010," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 35-44, April.
    35. R. D. Shelton & Loet Leydesdorff, 2012. "Publish or patent: Bibliometric evidence for empirical trade-offs in national funding strategies," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(3), pages 498-511, March.
    36. Vivek Kumar Singh & Ashraf Uddin & David Pinto, 2015. "Computer science research: the top 100 institutions in India and in the world," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(2), pages 529-553, August.
    37. Ludo Waltman & Nees Jan Eck, 2013. "Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(3), pages 699-716, September.
    38. Lie Yang & Lizi Han & Zhulei Chen & Jiabin Zhou & Jia Wang, 2015. "Growing trend of China’s contribution to haze research," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(1), pages 525-535, October.
    39. Yawen Zou & Manfred D. Laubichler, 2017. "Measuring the contributions of Chinese scholars to the research field of systems biology from 2005 to 2013," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(3), pages 1615-1631, March.
    40. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    41. Kearney, Colm, 2012. "Emerging markets research: Trends, issues and future directions," Emerging Markets Review, Elsevier, vol. 13(2), pages 159-183.
    42. Hagedoorn, John & Link, Albert N. & Vonortas, Nicholas S., 2000. "Research partnerships1," Research Policy, Elsevier, vol. 29(4-5), pages 567-586, April.
    43. Marcin Kozak & Lutz Bornmann & Loet Leydesdorff, 2015. "How have the Eastern European countries of the former Warsaw Pact developed since 1990? A bibliometric study," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1101-1117, February.
    44. Bárbara S. Lancho-Barrantes & Francisco J. Cantú-Ortiz, 2019. "Science in Mexico: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 499-517, February.
    45. Michel Zitt & Suzy Ramanana-Rahary & Elise Bassecoulard, 2003. "Correcting glasses help fair comparisons in international science landscape: Country indicators as a function of ISI database delineation," Scientometrics, Springer;Akadémiai Kiadó, vol. 56(2), pages 259-282, February.
    46. Luo, Yadong & Zhang, Huan, 2016. "Emerging Market MNEs: Qualitative Review and Theoretical Directions," Journal of International Management, Elsevier, vol. 22(4), pages 333-350.
    47. Leydesdorff, Loet & Wagner, Caroline, 2009. "Macro-level indicators of the relations between research funding and research output," Journal of Informetrics, Elsevier, vol. 3(4), pages 353-362.
    48. Bronwyn H. Hall & Albert N. Link & John T. Scott, 2003. "Universities as Research Partners," The Review of Economics and Statistics, MIT Press, vol. 85(2), pages 485-491, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hamdi A. Al-Jamimi & Galal M. BinMakhashen & Lutz Bornmann & Yousif Ahmed Al Wajih, 2023. "Saudi Arabia research: academic insights and trend analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5595-5627, October.
    2. Fadian Nur Aziz & Ali Roziqin & M. Jafar Loilatu & Tri Sulistyaningsih & Yana S. Hijri & Kismartini, 2024. "Research Trends of Creative Industries in Indonesia: A Bibliometric Analysis," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 15(1), pages 2571-2588, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hamdi A. Al-Jamimi & Galal M. BinMakhashen & Lutz Bornmann & Yousif Ahmed Al Wajih, 2023. "Saudi Arabia research: academic insights and trend analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5595-5627, October.
    2. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    3. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
    4. Zaida Chinchilla-Rodríguez & Cassidy R Sugimoto & Vincent Larivière, 2019. "Follow the leader: On the relationship between leadership and scholarly impact in international collaborations," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-18, June.
    5. Barbara S. Lancho-Barrantes & Hector G. Ceballos-Cancino & Francisco J. Cantu-Ortiz, 2021. "Comparing the efficiency of countries to assimilate and apply research investment," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(4), pages 1347-1369, August.
    6. Camil Demetrescu & Irene Finocchi & Andrea Ribichini & Marco Schaerf, 2022. "On computer science research and its temporal evolution," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4913-4938, August.
    7. Sara Amoroso & Alex Coad & Nicola Grassano, 2017. "European R&D networks: A snapshot from the 7th EU Framework Programme," JRC Working Papers on Corporate R&D and Innovation JRC107546, Joint Research Centre (Seville site).
    8. Dolata, Ulrich, 2014. "Märkte und Macht der Internetkonzerne: Konzentration - Konkurrenz - Innovationsstrategien," Research Contributions to Organizational Sociology and Innovation Studies, SOI Discussion Papers 2014-04, University of Stuttgart, Institute for Social Sciences, Department of Organizational Sociology and Innovation Studies.
    9. Gilsing, Victor & Nooteboom, Bart & Vanhaverbeke, Wim & Duysters, Geert & van den Oord, Ad, 2008. "Network embeddedness and the exploration of novel technologies: Technological distance, betweenness centrality and density," Research Policy, Elsevier, vol. 37(10), pages 1717-1731, December.
    10. Xiaoke Zhang, 2022. "Understanding innovation policy governance: A disaggregated approach," Review of Policy Research, Policy Studies Organization, vol. 39(3), pages 303-329, May.
    11. Bowns, Steven & Bradley, Ian & Knee, Paula & Williams, Fiona & Williams, Geoffrey, 2003. "Measuring the economic benefits from R&D: improvements in the MMI model of the United Kingdom National Measurement System," Research Policy, Elsevier, vol. 32(6), pages 991-1002, June.
    12. Yannis Caloghirou & Stavros Ioannides & Nicholas S. Vonortas, 2003. "Research Joint Ventures," Journal of Economic Surveys, Wiley Blackwell, vol. 17(4), pages 541-570, September.
    13. Fernando Muñoz-Bullón & Maria J. Sanchez-Bueno & Alfredo De Massis, 2020. "Combining Internal and External R&D: The Effects on Innovation Performance in Family and Nonfamily Firms," Entrepreneurship Theory and Practice, , vol. 44(5), pages 996-1031, September.
    14. Klessova, Svetlana & Engell, Sebastian & Thomas, Catherine, 2022. "Assessment of the advancement of market-upstream innovations and of the performance of research and innovation projects," Technovation, Elsevier, vol. 116(C).
    15. Helmers, Christian & Patnam, Manasa & Rau, P. Raghavendra, 2017. "Do board interlocks increase innovation? Evidence from a corporate governance reform in India," Journal of Banking & Finance, Elsevier, vol. 80(C), pages 51-70.
    16. De Silva, Muthu & Gokhberg, Leonid & Meissner, Dirk & Russo, Margherita, 2021. "Addressing societal challenges through the simultaneous generation of social and business values: A conceptual framework for science-based co-creation," Technovation, Elsevier, vol. 104(C).
    17. Daniel Nepelski & Giuseppe Piroli, 2018. "Organizational diversity and innovation potential of EU-funded research projects," The Journal of Technology Transfer, Springer, vol. 43(3), pages 615-639, June.
    18. Giovanni Cerulli & Giovanni Marin & Eleonora Pierucci & Bianca Potì, 2022. "Do company-owned academic patents influence firm performance? Evidence from the Italian industry," The Journal of Technology Transfer, Springer, vol. 47(1), pages 242-269, February.
    19. JongWuk Ahn & Hyundo Choi & Dong-hyun Oh, 2019. "Leveraging bridging universities to access international knowledge: Korean universities’ R&D internationalization," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 519-537, August.
    20. Enrique Acebo & José‐Ángel Miguel‐Dávila & Mariano Nieto, 2021. "External stakeholder engagement: Complementary and substitutive effects on firms' eco‐innovation," Business Strategy and the Environment, Wiley Blackwell, vol. 30(5), pages 2671-2687, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:127:y:2022:i:10:d:10.1007_s11192-022-04490-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.