IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i11d10.1007_s11192-024-05145-6.html
   My bibliography  Save this article

Interdisciplinarity and expert rating: an analysis based on faculty opinions

Author

Listed:
  • Shiji Chen

    (Hangzhou Dianzi University
    Université de Montréal)

  • Kaiqi Zhang

    (Hangzhou Dianzi University)

  • Junping Qiu

    (Hangzhou Dianzi University)

  • Jiaqi Chai

    (China Academy of Chinese Medical Sciences)

Abstract

The evaluation of interdisciplinary research is a crucial area of investigation in research evaluation. Many studies have examined the scientific impact of interdisciplinary research by assessing citation impact. However, there is a lack of research exploring the relationship between interdisciplinarity and peer review or expert opinions, particularly at the article level. This study aims to scrutinize the relationship between interdisciplinarity and both citation impact and Faculty Opinions recommendations (FORs) based on the dataset comprising 184,910 articles and 230,928 expert ratings from Faculty Opinions. The analysis yields two notable conclusions. Firstly, interdisciplinary research demonstrates a positive association with citations when considering composite interdisciplinarity indicators. Meanwhile, separate diversity measures reveal that variety and disparity positively impact citations, whereas balance exhibits a negative association with citations. Furthermore, a curvilinear relationship between interdisciplinarity and citations was identified, characterized by a U-shaped or J-shaped curve. Secondly, interdisciplinary research shows a negative association with FORs regarding composite interdisciplinarity indicators, indicating that experts are less likely to recommend interdisciplinary research. Analysis of separate diversity measures reveals that balance consistently exhibits a negative correlation with FORs, and disparity consistently displays a negative U-shaped association with FORs. In conclusion, although interdisciplinarity enhances citation impact, it appears to be underappreciated in Faculty Opinions.

Suggested Citation

  • Shiji Chen & Kaiqi Zhang & Junping Qiu & Jiaqi Chai, 2024. "Interdisciplinarity and expert rating: an analysis based on faculty opinions," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 6597-6628, November.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:11:d:10.1007_s11192-024-05145-6
    DOI: 10.1007/s11192-024-05145-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05145-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-024-05145-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Vincent Larivière & Yves Gingras, 2010. "On the relationship between interdisciplinarity and scientific impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(1), pages 126-131, January.
    2. Wang, Jian, 2016. "Knowledge creation in collaboration networks: Effects of tie configuration," Research Policy, Elsevier, vol. 45(1), pages 68-80.
    3. Alexis-Michel Mugabushaka & Anthi Kyriakou & Theo Papazoglou, 2016. "Bibliometric indicators of interdisciplinarity: the potential of the Leinster–Cobbold diversity indices to study disciplinary diversity," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 593-607, May.
    4. Vincent Larivière & Yves Gingras, 2010. "On the relationship between interdisciplinarity and scientific impact," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(1), pages 126-131, January.
    5. Andy Stirling, 2007. "A General Framework for Analysing Diversity in Science, Technology and Society," SPRU Working Paper Series 156, SPRU - Science Policy Research Unit, University of Sussex Business School.
    6. Thomas W. Steele & Jeffrey C. Stier, 2000. "The impact of interdisciplinary research in the environmental sciences: a forestry case study," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 51(5), pages 476-484.
    7. Lin Zhang & Ronald Rousseau & Wolfgang Glänzel, 2016. "Diversity of references as an indicator of the interdisciplinarity of journals: Taking similarity between subject fields into account," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(5), pages 1257-1265, May.
    8. Per Ahlgren & Bo Jarneving & Ronald Rousseau, 2003. "Requirements for a cocitation similarity measure, with special reference to Pearson's correlation coefficient," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 54(6), pages 550-560, April.
    9. Michèle Lamont & Grégoire Mallard & Joshua Guetzkow, 2006. "Beyond blind faith: overcoming the obstacles to interdisciplinary evaluation," Research Evaluation, Oxford University Press, vol. 15(1), pages 43-55, April.
    10. Jian Wang & Bart Thijs & Wolfgang Glänzel, 2015. "Interdisciplinarity and Impact: Distinct Effects of Variety, Balance, and Disparity," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-18, May.
    11. Ismael Rafols & Martin Meyer, 2010. "Diversity and network coherence as indicators of interdisciplinarity: case studies in bionanoscience," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 263-287, February.
    12. Chen, Shiji & Arsenault, Clément & Larivière, Vincent, 2015. "Are top-cited papers more interdisciplinary?," Journal of Informetrics, Elsevier, vol. 9(4), pages 1034-1046.
    13. Vieira, E.S. & Gomes, J.A.N.F., 2010. "Citations to scientific articles: Its distribution and dependence on the article features," Journal of Informetrics, Elsevier, vol. 4(1), pages 1-13.
    14. Tahamtan, Iman & Bornmann, Lutz, 2018. "Creativity in science and the link to cited references: Is the creative potential of papers reflected in their cited references?," Journal of Informetrics, Elsevier, vol. 12(3), pages 906-930.
    15. Wooldridge, Jeffrey M., 1999. "Distribution-free estimation of some nonlinear panel data models," Journal of Econometrics, Elsevier, vol. 90(1), pages 77-97, May.
    16. Rinia, E. J. & van Leeuwen, Th. N. & van Vuren, H. G. & van Raan, A. F. J., 2001. "Influence of interdisciplinarity on peer-review and bibliometric evaluations in physics research," Research Policy, Elsevier, vol. 30(3), pages 357-361, March.
    17. Jonathan M. Levitt & Mike Thelwall, 2008. "Is multidisciplinary research more highly cited? A macrolevel study," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(12), pages 1973-1984, October.
    18. Maxime Rivest & Etienne Vignola-Gagné & Éric Archambault, 2021. "Article-level classification of scientific publications: A comparison of deep learning, direct citation and bibliographic coupling," PLOS ONE, Public Library of Science, vol. 16(5), pages 1-18, May.
    19. Jonathan M. Levitt & Mike Thelwall, 2009. "The most highly cited Library and Information Science articles: Interdisciplinarity, first authors and citation patterns," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(1), pages 45-67, January.
    20. Loet Leydesdorff, 2018. "Diversity and interdisciplinarity: how can one distinguish and recombine disparity, variety, and balance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2113-2121, September.
    21. Bornmann, Lutz & Leydesdorff, Loet, 2013. "The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000," Journal of Informetrics, Elsevier, vol. 7(2), pages 286-291.
    22. Lindell Bromham & Russell Dinnage & Xia Hua, 2016. "Interdisciplinary research has consistently lower funding success," Nature, Nature, vol. 534(7609), pages 684-687, June.
    23. Lutz Bornmann & Hans-Dieter Daniel, 2006. "Selecting scientific excellence through committee peer review - A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicants," Scientometrics, Springer;Akadémiai Kiadó, vol. 68(3), pages 427-440, September.
    24. Lutz Bornmann & Robin Haunschild, 2018. "Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-12, May.
    25. Chen, Shiji & Qiu, Junping & Arsenault, Clément & Larivière, Vincent, 2021. "Exploring the interdisciplinarity patterns of highly cited papers," Journal of Informetrics, Elsevier, vol. 15(1).
    26. Ludo Waltman & Rodrigo Costas, 2014. "F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 433-445, March.
    27. Moed, Henk F., 2010. "Measuring contextual citation impact of scientific journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 265-277.
    28. Shiji Chen & Yanhui Song & Fei Shu & Vincent Larivière, 2022. "Interdisciplinarity and impact: the effects of the citation time window," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2621-2642, May.
    29. Ed J. Rinia & Thed N. van Leeuwen & Anthony F. J. van Raan, 2002. "Impact measures of interdisciplinary research in physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 241-248, February.
    30. Lin Zhang & Beibei Sun & Lidan Jiang & Ying Huang, 2021. "On the relationship between interdisciplinarity and impact: Distinct effects on academic and broader impact [A Comparison of Two Approaches for Measuring Interdisciplinary Research Output: The Disc," Research Evaluation, Oxford University Press, vol. 30(3), pages 256-268.
    31. Jian Du & Xiaoli Tang & Yishan Wu, 2016. "The effects of research level and article type on the differences between citation metrics and F1000 recommendations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(12), pages 3008-3021, December.
    32. Alfredo Yegros-Yegros & Ismael Rafols & Pablo D’Este, 2015. "Does Interdisciplinary Research Lead to Higher Citation Impact? The Different Effect of Proximal and Distal Interdisciplinarity," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-21, August.
    33. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    34. Alan L. Porter & Ismael Rafols, 2009. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 719-745, December.
    35. Cummings, Jonathon N. & Kiesler, Sara, 2007. "Coordination costs and project outcomes in multi-university collaborations," Research Policy, Elsevier, vol. 36(10), pages 1620-1634, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shiji Chen & Yanhui Song & Fei Shu & Vincent Larivière, 2022. "Interdisciplinarity and impact: the effects of the citation time window," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2621-2642, May.
    2. Chen, Shiji & Qiu, Junping & Arsenault, Clément & Larivière, Vincent, 2021. "Exploring the interdisciplinarity patterns of highly cited papers," Journal of Informetrics, Elsevier, vol. 15(1).
    3. Fei Shu & Jesse David Dinneen & Shiji Chen, 2022. "Measuring the disparity among scientific disciplines using Library of Congress Subject Headings," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3613-3628, June.
    4. Giulio Giacomo Cantone, 2024. "How to measure interdisciplinary research? A systemic design for the model of measurement," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 4937-4982, August.
    5. Hou, Jianhua & Li, Hao & Zhang, Yang, 2024. "Influence of interdisciplinarity of scientific papers on the durability of citation diffusion: A perspective from citation discontinuance," Journal of Informetrics, Elsevier, vol. 18(3).
    6. Zhang, Yang & Wang, Yang & Du, Haifeng & Havlin, Shlomo, 2024. "Delayed citation impact of interdisciplinary research," Journal of Informetrics, Elsevier, vol. 18(1).
    7. Chen, Shiji & Arsenault, Clément & Larivière, Vincent, 2015. "Are top-cited papers more interdisciplinary?," Journal of Informetrics, Elsevier, vol. 9(4), pages 1034-1046.
    8. Qing Ke, 2023. "Interdisciplinary research and technological impact: evidence from biomedicine," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2035-2077, April.
    9. Andrea Bonaccorsi & Nicola Melluso & Francesco Alessandro Massucci, 2022. "Exploring the antecedents of interdisciplinarity at the European Research Council: a topic modeling approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 6961-6991, December.
    10. Jingjing Ren & Fang Wang & Minglu Li, 2023. "Dynamics and characteristics of interdisciplinary research in scientific breakthroughs: case studies of Nobel-winning research in the past 120 years," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4383-4419, August.
    11. Xin Liu & Yi Bu & Ming Li & Jiang Li, 2024. "Monodisciplinary collaboration disrupts science more than multidisciplinary collaboration," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 75(1), pages 59-78, January.
    12. Alfonso Ávila-Robinson & Cristian Mejia & Shintaro Sengoku, 2021. "Are bibliometric measures consistent with scientists’ perceptions? The case of interdisciplinarity in research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7477-7502, September.
    13. Xiaojing Cai & Xiaozan Lyu & Ping Zhou, 2023. "The relationship between interdisciplinarity and citation impact—a novel perspective on citation accumulation," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-12, December.
    14. Keisuke Okamura, 2019. "Interdisciplinarity revisited: evidence for research impact and dynamism," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-9, December.
    15. Lina Xu & Steven Dellaportas & Zhiqiang Yang & Jin Wang, 2023. "More on the relationship between interdisciplinary accounting research and citation impact," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 63(4), pages 4779-4803, December.
    16. Hongyu Zhou & Raf Guns & Tim C. E. Engels, 2022. "Are social sciences becoming more interdisciplinary? Evidence from publications 1960–2014," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(9), pages 1201-1221, September.
    17. Francesco Giovanni Avallone & Alberto Quagli & Paola Ramassa, 2022. "Interdisciplinary research by accounting scholars: An exploratory study," FINANCIAL REPORTING, FrancoAngeli Editore, vol. 2022(2), pages 5-34.
    18. Seolmin Yang & So Young Kim, 2023. "Knowledge-integrated research is more disruptive when supported by homogeneous funding sources: a case of US federally funded research in biomedical and life sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3257-3282, June.
    19. Leydesdorff, Loet & Wagner, Caroline S. & Bornmann, Lutz, 2019. "Interdisciplinarity as diversity in citation patterns among journals: Rao-Stirling diversity, relative variety, and the Gini coefficient," Journal of Informetrics, Elsevier, vol. 13(1), pages 255-269.
    20. Lina Xu & Steven Dellaportas & Jin Wang, 2022. "A study of interdisciplinary accounting research: analysing the diversity of cited references," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 62(2), pages 2131-2162, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:11:d:10.1007_s11192-024-05145-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.