IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v127y2022i10d10.1007_s11192-022-04490-8.html
   My bibliography  Save this article

Use of bibliometrics for research evaluation in emerging markets economies: a review and discussion of bibliometric indicators

Author

Listed:
  • Hamdi A. Al-Jamimi

    (Research Institute, King Fahd University of Petroleum & Minerals)

  • Galal M. BinMakhashen

    (Research Institute, King Fahd University of Petroleum & Minerals)

  • Lutz Bornmann

    (Max Planck Institute for Solid State Research
    Administrative Headquarters of the Max Planck Society)

Abstract

The governments of most ‘emerging market’ (EM) countries (e.g., India, Mexico, and Brazil) have invested significantly in scientific research over the past decades. These investments are reflected in the countries’ research volume, impact, and international collaboration activities. The EMs’ research support justification delivers the need to evaluate their research performance at multi-levels (country, institution, and scholar), e.g., by using indicators of research application inclinations. Bibliometric studies are increasingly requested by research and development (R&D) policymakers, funding agencies, and other relevant actors in these countries. The results of the studies are used to develop research directions, set research policies, and satisfy management needs (at universities). In this paper, we review the contributions of previous (bibliometric) studies to measuring research performance in EM countries. We identified 132 indicators used in the studies and classified them into three categories: productivity, impact, and collaboration. Some indicators have been more frequently used than others for measuring the performance of EM countries. These indicators might be the most important ones. For example, research leadership indicators (focusing on corresponding authors of publications) might be more important than other indicators measuring the research productivity of EM countries since they highlight the ‘actual’ contribution of national scientists. Collaboration indicators measuring national/international collaborative publications are also especially relevant for EM countries; their results can be used to sharpen expertise and exchange of professional knowledge. The overview presented in this paper can be used for the planning and realization of future studies on research in EM countries.

Suggested Citation

  • Hamdi A. Al-Jamimi & Galal M. BinMakhashen & Lutz Bornmann, 2022. "Use of bibliometrics for research evaluation in emerging markets economies: a review and discussion of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5879-5930, October.
  • Handle: RePEc:spr:scient:v:127:y:2022:i:10:d:10.1007_s11192-022-04490-8
    DOI: 10.1007/s11192-022-04490-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-022-04490-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-022-04490-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kearney, Colm, 2012. "Emerging markets research: Trends, issues and future directions," Emerging Markets Review, Elsevier, vol. 13(2), pages 159-183.
    2. Qiang Yao & Peng-Hui Lyu & Lian-Ping Yang & Lan Yao & Zhi-Yong Liu, 2014. "Current performance and future trends in health care sciences and services research," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 751-779, October.
    3. Luna-Morales Maria Elena & Luna-Morales Evelia & Pérez-Angón Miguel Ángel, 2021. "Influence of the international collaboration in the field of metric studies of science and technology: the case of Mexico (1971–2018)," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(3), pages 2485-2511, March.
    4. Gangan Prathap, 2010. "The 100 most prolific economists using the p-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 167-172, July.
    5. Jun-Ying Fu & Xu Zhang & Yun-Hua Zhao & Mu-Hsuan Huang & Dar-Zen Chen, 2011. "Bibliometric analysis of complementary and alternative medicine research over three decades," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 617-626, August.
    6. Gustavo Cattelan Nobre & Elaine Tavares, 2017. "Scientific literature analysis on big data and internet of things applications on circular economy: a bibliometric study," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 463-492, April.
    7. Minsoo Choi & Heejin Lee & Hanah Zoo, 2021. "Scientific knowledge production and research collaboration between Australia and South Korea: patterns and dynamics based on co-authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 683-706, January.
    8. Koski, Timo & Sandström, Erik & Sandström, Ulf, 2016. "Towards field-adjusted production: Estimating research productivity from a zero-truncated distribution," Journal of Informetrics, Elsevier, vol. 10(4), pages 1143-1152.
    9. Hagedoorn, John & Link, Albert N. & Vonortas, Nicholas S., 2000. "Research partnerships1," Research Policy, Elsevier, vol. 29(4-5), pages 567-586, April.
    10. I. Jarić & J. Gessner, 2012. "Analysis of publications on sturgeon research between 1996 and 2010," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 715-735, February.
    11. Lutz Bornmann, 2014. "How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature," Research Evaluation, Oxford University Press, vol. 23(2), pages 166-173.
    12. Robert J. W. Tijssen & Jos Winnink, 2016. "Twenty-first century macro-trends in the institutional fabric of science: bibliometric monitoring and analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2181-2194, December.
    13. Weishu Liu & Mengdi Gu & Guangyuan Hu & Chao Li & Huchang Liao & Li Tang & Philip Shapira, 2014. "Profile of developments in biomass-based bioenergy research: a 20-year perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 507-521, May.
    14. Marcin Kozak & Lutz Bornmann & Loet Leydesdorff, 2015. "How have the Eastern European countries of the former Warsaw Pact developed since 1990? A bibliometric study," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1101-1117, February.
    15. Gangan Prathap, 2011. "The Energy–Exergy–Entropy (or EEE) sequences in bibliometric assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 515-524, June.
    16. Muzammil Tahira & Rose Alinda Alias & Aryati Bakri & A. Abrizah, 2016. "Meso-level institutional and journal related indices for Malaysian engineering research," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 521-535, May.
    17. Muhammad Farooq & Muhammad Asim & Muhammad Imran & Shahid Imran & Jameel Ahmad & Muhammad Rizwan Younis, 2018. "Mapping past, current and future energy research trend in Pakistan: a scientometric assessment," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1733-1753, December.
    18. V. A. Markusova & A. N. Libkind & A. E. Varshavsly & C. N. M. Jansz, 2012. "Research performance and collaboration in the Novosibirsk region," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 513-526, May.
    19. Xiao Han T Zeng & Jordi Duch & Marta Sales-Pardo & João A G Moreira & Filippo Radicchi & Haroldo V Ribeiro & Teresa K Woodruff & Luís A Nunes Amaral, 2016. "Differences in Collaboration Patterns across Discipline, Career Stage, and Gender," PLOS Biology, Public Library of Science, vol. 14(11), pages 1-19, November.
    20. Adams, Jonathan, 2018. "Information and misinformation in bibliometric time-trend analysis," Journal of Informetrics, Elsevier, vol. 12(4), pages 1063-1071.
    21. Fu-Chiang Yang & Rui-Hsin Kao & Yi-Tui Chen & Yueh-Fei Ho & Cheng-Chung Cho & Shi-Wei Huang, 2018. "A Common Weight Approach to Construct Composite Indicators: The Evaluation of Fourteen Emerging Markets," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 137(2), pages 463-479, June.
    22. Jiang Tan & Hui-Zhen Fu & Yuh-Shan Ho, 2014. "A bibliometric analysis of research on proteomics in Science Citation Index Expanded," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1473-1490, February.
    23. Hamid Darvish & Yaşar Tonta, 2016. "Diffusion of nanotechnology knowledge in Turkey and its network structure," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 569-592, May.
    24. Bárbara S. Lancho-Barrantes & Francisco J. Cantú-Ortiz, 2019. "Science in Mexico: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 499-517, February.
    25. Pardeep Sud & Mike Thelwall, 2014. "Evaluating altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1131-1143, February.
    26. Elio Atenógenes Villaseñor & Ricardo Arencibia-Jorge & Humberto Carrillo-Calvet, 2017. "Multiparametric characterization of scientometric performance profiles assisted by neural networks: a study of Mexican higher education institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 77-104, January.
    27. Shakil Ahmad & Yasir Javed & Shabir Hussain Khahro & Arslan Shahid, 2020. "Research Contribution of the Oldest Seat of Higher Learning in Pakistan: A Bibliometric Analysis of University of the Punjab," Publications, MDPI, vol. 8(3), pages 1-15, September.
    28. Michel Zitt & Suzy Ramanana-Rahary & Elise Bassecoulard, 2003. "Correcting glasses help fair comparisons in international science landscape: Country indicators as a function of ISI database delineation," Scientometrics, Springer;Akadémiai Kiadó, vol. 56(2), pages 259-282, February.
    29. Bronwyn H. Hall & Albert N. Link & John T. Scott, 2003. "Universities as Research Partners," The Review of Economics and Statistics, MIT Press, vol. 85(2), pages 485-491, May.
    30. Har Kaur & B. M. Gupta, 2010. "Mapping of dental science research in India: a scientometric analysis of India’s research output, 1999–2008," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 361-376, October.
    31. Chang-Jing Ji & Xiao-Yi Li & Yu-Jie Hu & Xiang-Yu Wang & Bao-Jun Tang, 2019. "Research on carbon price in emissions trading scheme: a bibliometric analysis," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 99(3), pages 1381-1396, December.
    32. K.C. Garg & P. Padhi, 2001. "A study of collaboration in laser science and technology," Scientometrics, Springer;Akadémiai Kiadó, vol. 51(2), pages 415-427, June.
    33. Luo, Yadong & Zhang, Huan, 2016. "Emerging Market MNEs: Qualitative Review and Theoretical Directions," Journal of International Management, Elsevier, vol. 22(4), pages 333-350.
    34. Hans Pohl & Jason E. Lane, 2018. "Research contributions of international branch campuses to the scientific wealth of academically developing countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1719-1734, September.
    35. Veronica Perez-Cabezas & Carmen Ruiz-Molinero & Ines Carmona-Barrientos & Enrique Herrera-Viedma & Manuel J. Cobo & Jose A. Moral-Munoz, 2018. "Highly cited papers in rheumatology: identification and conceptual analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 555-568, July.
    36. Liang Zhang & Sisi Li & Hugo A. Loáiciga & Yanhua Zhuang & Yun Du, 2015. "Opportunities and challenges of interbasin water transfers: a literature review with bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(1), pages 279-294, October.
    37. Zaida Chinchilla-Rodríguez & Grisel Zacca-González & Benjamín Vargas-Quesada & Félix Moya-Anegón, 2015. "Latin American scientific output in Public Health: combined analysis using bibliometric, socioeconomic and health indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 609-628, January.
    38. Abdulkader A. Murad & Dimitar T. Tomov, 2012. "Institutionalization and internationalization of research on the applications of the geographical information systems in health planning," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(1), pages 143-158, April.
    39. Leydesdorff, Loet & Wagner, Caroline, 2009. "Macro-level indicators of the relations between research funding and research output," Journal of Informetrics, Elsevier, vol. 3(4), pages 353-362.
    40. Nicholas V. Olijnyk, 2015. "A quantitative examination of the intellectual profile and evolution of information security from 1965 to 2015," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(2), pages 883-904, November.
    41. Bronwyn H. Hall & Albert N. Link & John T. Scott, 2003. "Universities as Research Partners," The Review of Economics and Statistics, MIT Press, vol. 85(2), pages 485-491, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hamdi A. Al-Jamimi & Galal M. BinMakhashen & Lutz Bornmann & Yousif Ahmed Al Wajih, 2023. "Saudi Arabia research: academic insights and trend analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5595-5627, October.
    2. Mohammed Abdul Fasi, 2024. "Triple Helix Collaboration: Uncovering Research Opportunities Among Industry, Academia, and Researchers—A Case Study of Saudi Aramco’s Patent Portfolio Analysis," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 15(3), pages 14486-14520, September.
    3. Fadian Nur Aziz & Ali Roziqin & M. Jafar Loilatu & Tri Sulistyaningsih & Yana S. Hijri & Kismartini, 2024. "Research Trends of Creative Industries in Indonesia: A Bibliometric Analysis," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 15(1), pages 2571-2588, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hamdi A. Al-Jamimi & Galal M. BinMakhashen & Lutz Bornmann & Yousif Ahmed Al Wajih, 2023. "Saudi Arabia research: academic insights and trend analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5595-5627, October.
    2. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    3. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
    4. Zaida Chinchilla-Rodríguez & Cassidy R Sugimoto & Vincent Larivière, 2019. "Follow the leader: On the relationship between leadership and scholarly impact in international collaborations," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-18, June.
    5. Barbara S. Lancho-Barrantes & Hector G. Ceballos-Cancino & Francisco J. Cantu-Ortiz, 2021. "Comparing the efficiency of countries to assimilate and apply research investment," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(4), pages 1347-1369, August.
    6. Camil Demetrescu & Irene Finocchi & Andrea Ribichini & Marco Schaerf, 2022. "On computer science research and its temporal evolution," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4913-4938, August.
    7. Manning, Stephan, 2017. "The rise of project network organizations: Building core teams and flexible partner pools for interorganizational projects," Research Policy, Elsevier, vol. 46(8), pages 1399-1415.
    8. Sara Amoroso & Alex Coad & Nicola Grassano, 2017. "European R&D networks: A snapshot from the 7th EU Framework Programme," JRC Working Papers on Corporate R&D and Innovation JRC107546, Joint Research Centre (Seville site).
    9. Dolata, Ulrich, 2014. "Märkte und Macht der Internetkonzerne: Konzentration - Konkurrenz - Innovationsstrategien," Research Contributions to Organizational Sociology and Innovation Studies, SOI Discussion Papers 2014-04, University of Stuttgart, Institute for Social Sciences, Department of Organizational Sociology and Innovation Studies.
    10. Tamara Krajna & Jelka Petrak, 2019. "Croatian Highly Cited Papers," Interdisciplinary Description of Complex Systems - scientific journal, Croatian Interdisciplinary Society Provider Homepage: http://indecs.eu, vol. 17(3-B), pages 684-696.
    11. Nobuko Miyairi & Han-Wen Chang, 2012. "Bibliometric characteristics of highly cited papers from Taiwan, 2000–2009," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(1), pages 197-205, July.
    12. Hala Abou-Ali & Mohammed Belhaj, 2008. "Cost Benefit Analysis of Desert Locusts Control: A Multicountry Perspective," Working Papers 801, Economic Research Forum, revised 01 Jan 2008.
    13. Lee, Young Hoon & Kim, YoungJun, 2016. "Analyzing interaction in R&D networks using the Triple Helix method: Evidence from industrial R&D programs in Korean government," Technological Forecasting and Social Change, Elsevier, vol. 110(C), pages 93-105.
    14. Moussa, Bachar & Varsakelis, Nikos C., 2017. "International patenting: An application of network analysis," The Journal of Economic Asymmetries, Elsevier, vol. 15(C), pages 48-55.
    15. Gilsing, Victor & Nooteboom, Bart & Vanhaverbeke, Wim & Duysters, Geert & van den Oord, Ad, 2008. "Network embeddedness and the exploration of novel technologies: Technological distance, betweenness centrality and density," Research Policy, Elsevier, vol. 37(10), pages 1717-1731, December.
    16. Xiaoke Zhang, 2022. "Understanding innovation policy governance: A disaggregated approach," Review of Policy Research, Policy Studies Organization, vol. 39(3), pages 303-329, May.
    17. João José M. Ferreira & Cristina I. Fernandes & Vanessa Ratten, 2016. "A co-citation bibliometric analysis of strategic management research," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(1), pages 1-32, October.
    18. Seo, Hangyeol & Chung, Yanghon & Yoon, Hyungseok (David), 2017. "R&D cooperation and unintended innovation performance: Role of appropriability regimes and sectoral characteristics," Technovation, Elsevier, vol. 66, pages 28-42.
    19. Steinmueller, W. Edward, 2010. "Economics of Technology Policy," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 2, chapter 0, pages 1181-1218, Elsevier.
    20. Andrew A. Toole & Dirk Czarnitzki & Christian Rammer, 2015. "University research alliances, absorptive capacity, and the contribution of startups to employment growth," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 24(5), pages 532-549, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:127:y:2022:i:10:d:10.1007_s11192-022-04490-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.