IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i9d10.1007_s11192-024-05129-6.html
   My bibliography  Save this article

Spotting potential reviewers for interdisciplinary research: insights on active reviewers from Publons

Author

Listed:
  • Lin Zhang

    (Wuhan University
    Wuhan University
    KU Leuven)

  • Ziyi Tu

    (Wuhan University
    Wuhan University)

  • Yifei Yu

    (Wuhan University
    Wuhan University)

  • Yuanyuan Shang

    (Chinese Academy of Social Sciences Evaluation Studies)

  • Ying Huang

    (Wuhan University
    Wuhan University
    KU Leuven)

Abstract

The increasing significance of interdisciplinary research poses a challenge in evaluating its quality and emphasizes the need for academic journals to choose appropriate reviewers for interdisciplinary research. Several studies have focused on the problems in reviewing interdisciplinary research, but fewer studies on the challenges of recruiting suitable reviewers for interdisciplinary research. Our study collects a specific group of active interdisciplinary reviewers who were awarded the “Top 1% reviewers in Cross-field” by Publons—a recognition for reviewers conducting the highest number of reviews in corresponding fields. We aim to examine the interdisciplinary backgrounds and intrinsic interactivity of these active interdisciplinary reviewers in various academic activities, alongside exploring the countries, institutions, and research fields they represent. Our findings shed light on the uneven distribution of active interdisciplinary reviewers among countries and institutions. We also observe that active interdisciplinary reviewers often possess high interdisciplinarity in their research activities, and this interdisciplinarity increases as their reviewing and research experience grows. Notably, moderately experienced reviewers (with 6–10 years of reviewing experience) are the primary force of active interdisciplinary reviewers. We also advocate for inviting reviewers with varied levels of research and reviewing experience commensurate with the interdisciplinarity or complexity of the research. Moreover, active interdisciplinary reviewers demonstrate a strong disciplinary similarity and correlation among their research, reviewing, and editorial activities. Given the mounting pressure and challenges associated with reviewing interdisciplinary research, we propose several recommendations based on our analysis.

Suggested Citation

  • Lin Zhang & Ziyi Tu & Yifei Yu & Yuanyuan Shang & Ying Huang, 2024. "Spotting potential reviewers for interdisciplinary research: insights on active reviewers from Publons," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(9), pages 5533-5556, September.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:9:d:10.1007_s11192-024-05129-6
    DOI: 10.1007/s11192-024-05129-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05129-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-024-05129-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ismael Rafols & Martin Meyer, 2010. "Diversity and network coherence as indicators of interdisciplinarity: case studies in bionanoscience," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 263-287, February.
    2. Ulf Sandström & Martin Hällsten, 2008. "Persistent nepotism in peer-review," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 175-189, February.
    3. Marco Seeber & Jef Vlegels & Mattia Cattaneo, 2022. "Conditions that do or do not disadvantage interdisciplinary research proposals in project evaluation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(8), pages 1106-1126, August.
    4. repec:pal:palcom:v:2016:y:2016:i:palcomms201655:p:16055- is not listed on IDEAS
    5. Michail Kovanis & Raphaël Porcher & Philippe Ravaud & Ludovic Trinquart, 2016. "The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise," PLOS ONE, Public Library of Science, vol. 11(11), pages 1-14, November.
    6. Tao Jia & Dashun Wang & Boleslaw K. Szymanski, 2017. "Quantifying patterns of research-interest evolution," Nature Human Behaviour, Nature, vol. 1(4), pages 1-7, April.
    7. Niccolò Casnici & Francisco Grimaldo & Nigel Gilbert & Flaminio Squazzoni, 2017. "Attitudes of referees in a multidisciplinary journal: An empirical analysis," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(7), pages 1763-1771, July.
    8. Lin Zhang & Yuanyuan Shang & Ying Huang & Gunnar Sivertsen, 2022. "Gender differences among active reviewers: an investigation based on publons," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 145-179, January.
    9. Pautasso, Marco & Pautasso, Cesare, 2010. "Peer Reviewing Interdisciplinary Papers," European Review, Cambridge University Press, vol. 18(2), pages 227-237, May.
    10. Yordan Kalmukov, 2020. "An algorithm for automatic assignment of reviewers to papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 1811-1850, September.
    11. Lindell Bromham & Russell Dinnage & Xia Hua, 2016. "Interdisciplinary research has consistently lower funding success," Nature, Nature, vol. 534(7609), pages 684-687, June.
    12. repec:pal:palcom:v:2016:y:2016:i:palcomms201617:p:16017- is not listed on IDEAS
    13. Lawrence Smolinsky & Daniel S. Sage & Aaron J. Lercher & Aaron Cao, 2021. "Citations versus expert opinions: citation analysis of featured reviews of the American Mathematical Society," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 3853-3870, May.
    14. Xiaoyu Liu & Xuefeng Wang & Donghua Zhu, 2022. "Reviewer recommendation method for scientific research proposals: a case for NSFC," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3343-3366, June.
    15. Veronica Boix Mansilla, 2006. "Assessing expert interdisciplinary work at the frontier: an empirical exploration," Research Evaluation, Oxford University Press, vol. 15(1), pages 17-29, April.
    16. Charles W. Fox, 2017. "Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 465-477, October.
    17. Zhang, Guangyao & Xu, Shenmeng & Sun, Yao & Jiang, Chunlin & Wang, Xianwen, 2022. "Understanding the peer review endeavor in scientific publishing," Journal of Informetrics, Elsevier, vol. 16(2).
    18. Gabriele Bammer, 2016. "What constitutes appropriate peer review for interdisciplinary research?," Palgrave Communications, Palgrave Macmillan, vol. 2(1), pages 1-5, December.
    19. Tom McLeish & Veronica Strang, 2016. "Evaluating interdisciplinary research: the elephant in the peer-reviewers’ room," Palgrave Communications, Palgrave Macmillan, vol. 2(1), pages 1-8, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Seeber, Marco & Alon, Ilan & Pina, David G. & Piro, Fredrik Niclas & Seeber, Michele, 2022. "Predictors of applying for and winning an ERC Proof-of-Concept grant: An automated machine learning model," Technological Forecasting and Social Change, Elsevier, vol. 184(C).
    2. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    3. Lina Xu & Steven Dellaportas & Zhiqiang Yang & Jin Wang, 2023. "More on the relationship between interdisciplinary accounting research and citation impact," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 63(4), pages 4779-4803, December.
    4. Yu, Xiaoyao & Szymanski, Boleslaw K. & Jia, Tao, 2021. "Become a better you: Correlation between the change of research direction and the change of scientific performance," Journal of Informetrics, Elsevier, vol. 15(3).
    5. Balázs Győrffy & Andrea Magda Nagy & Péter Herman & Ádám Török, 2018. "Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 409-426, October.
    6. Flurina Schneider & Zarina Patel & Katsia Paulavets & Tobias Buser & Jacqueline Kado & Stefanie Burkhart, 2023. "Fostering transdisciplinary research for sustainability in the Global South: Pathways to impact for funding programmes," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-11, December.
    7. Bianchi, Federico & Grimaldo, Francisco & Squazzoni, Flaminio, 2019. "The F3-index. Valuing reviewers for scholarly journals," Journal of Informetrics, Elsevier, vol. 13(1), pages 78-86.
    8. Feng Shi & James Evans, 2023. "Surprising combinations of research contents and contexts are related to impact and emerge with scientific outsiders from distant disciplines," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    9. Fei Shu & Jesse David Dinneen & Shiji Chen, 2022. "Measuring the disparity among scientific disciplines using Library of Congress Subject Headings," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3613-3628, June.
    10. Xiaojing Cai & Xiaozan Lyu & Ping Zhou, 2023. "The relationship between interdisciplinarity and citation impact—a novel perspective on citation accumulation," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-12, December.
    11. Yongchao Ma & Ying Teng & Zhongzhun Deng & Li Liu & Yi Zhang, 2023. "Does writing style affect gender differences in the research performance of articles?: An empirical study of BERT-based textual sentiment analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2105-2143, April.
    12. Weixi Xie & Pengfei Jia & Guangyao Zhang & Xianwen Wang, 2024. "Are reviewer scores consistent with citations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 4721-4740, August.
    13. Andrea Bonaccorsi & Nicola Melluso & Francesco Alessandro Massucci, 2022. "Exploring the antecedents of interdisciplinarity at the European Research Council: a topic modeling approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 6961-6991, December.
    14. Shogo Katoh & Rick (H.L.) Aalbers & Shintaro Sengoku, 2021. "Effects and Interactions of Researcher’s Motivation and Personality in Promoting Interdisciplinary and Transdisciplinary Research," Sustainability, MDPI, vol. 13(22), pages 1-19, November.
    15. Sun, Zhuanlan, 2024. "Textual features of peer review predict top-cited papers: An interpretable machine learning perspective," Journal of Informetrics, Elsevier, vol. 18(2).
    16. Giulio Giacomo Cantone, 2024. "How to measure interdisciplinary research? A systemic design for the model of measurement," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 4937-4982, August.
    17. Bammer, Gabriele, 2018. "Strengthening community operational research through exchange of tools and strategic alliances," European Journal of Operational Research, Elsevier, vol. 268(3), pages 1168-1177.
    18. García, J.A. & Montero-Parodi, J.J. & Rodriguez-Sánchez, Rosa & Fdez-Valdivia, J., 2023. "How to motivate a reviewer with a present bias to work harder," Journal of Informetrics, Elsevier, vol. 17(4).
    19. Guo, Liying & Wang, Yang & Li, Meiling, 2024. "Exploration, exploitation and funding success: Evidence from junior scientists supported by the Chinese Young Scientists Fund," Journal of Informetrics, Elsevier, vol. 18(2).
    20. Oviedo-García, M. Ángeles, 2016. "Tourism research quality: Reviewing and assessing interdisciplinarity," Tourism Management, Elsevier, vol. 52(C), pages 586-592.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:9:d:10.1007_s11192-024-05129-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.