IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v127y2022i12d10.1007_s11192-022-04310-z.html
   My bibliography  Save this article

Toward greater consistency and validity in measuring interdisciplinarity: a systematic and conceptual evaluation

Author

Listed:
  • Sander Zwanenburg

    (University of Otago)

  • Maryam Nakhoda

    (University of Otago)

  • Peter Whigham

    (University of Otago)

Abstract

While interdisciplinary research (IDR) has attracted much attention, this has not yet resulted in a coherent body of knowledge of interdisciplinarity. One of the impediments is a lack of consensus on its conceptualization and measurement. Some of the proposed measures have shown to misalign empirically, meaning that conclusions about IDR can differ across measures. To clarify this disagreement conceptually, and to stimulate better coherence in measurement, this paper starts with a review of the IDR definitions. From a synthesis of these definitions, we provide a conceptual definition and a logical structure of the construct, and derive evaluation criteria for its measures. We use these to evaluate 21 measures of IDR. The results show that measures vary widely in meeting the criteria, which can explain some of the observed inconsistencies in earlier studies. We discuss the most common limitations and present empirical analyses to gauge their severity. We present several suggestions for future measurement of the interdisciplinarity of research. We hope that with these suggestions, researchers can draw more consistent conclusions, aiding in the development of a coherent body of knowledge of this ever-important phenomenon.

Suggested Citation

  • Sander Zwanenburg & Maryam Nakhoda & Peter Whigham, 2022. "Toward greater consistency and validity in measuring interdisciplinarity: a systematic and conceptual evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7769-7788, December.
    • Handle: RePEc:spr:scient:v:127:y:2022:i:12:d:10.1007_s11192-022-04310-z
    DOI: 10.1007/s11192-022-04310-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-022-04310-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-022-04310-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Xuefeng Wang & Zhinan Wang & Ying Huang & Yun Chen & Yi Zhang & Huichao Ren & Rongrong Li & Jinhui Pang, 2017. "Measuring interdisciplinarity of a research system: detecting distinction between publication categories and citation categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 2023-2039, June.
    2. Fernanda Morillo & María Bordons & Isabel Gómez, 2003. "Interdisciplinarity in science: A tentative typology of disciplines and research areas," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 54(13), pages 1237-1249, November.
    3. Bart Thijs & Lin Zhang & Wolfgang Glänzel, 2015. "Bibliographic coupling and hierarchical clustering for the validation and improvement of subject-classification schemes," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1453-1467, December.
    4. Jian Wang & Bart Thijs & Wolfgang Glänzel, 2015. "Interdisciplinarity and Impact: Distinct Effects of Variety, Balance, and Disparity," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-18, May.
    5. Mario Karlovčec & Dunja Mladenić, 2015. "Interdisciplinarity of scientific fields and its evolution based on graph of project collaboration and co-authoring," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 433-454, January.
    6. Huutoniemi, Katri & Klein, Julie Thompson & Bruun, Henrik & Hukkinen, Janne, 2010. "Analyzing interdisciplinarity: Typology and indicators," Research Policy, Elsevier, vol. 39(1), pages 79-88, February.
    7. Ed J. Rinia & Thed N. van Leeuwen & Anthony F. J. van Raan, 2002. "Impact measures of interdisciplinary research in physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 241-248, February.
    8. Alexis-Michel Mugabushaka & Anthi Kyriakou & Theo Papazoglou, 2016. "Bibliometric indicators of interdisciplinarity: the potential of the Leinster–Cobbold diversity indices to study disciplinary diversity," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 593-607, May.
    9. Rhoten, Diana & Pfirman, Stephanie, 2007. "Women in interdisciplinary science: Exploring preferences and consequences," Research Policy, Elsevier, vol. 36(1), pages 56-75, February.
    10. Joachim Schummer, 2004. "Multidisciplinarity, interdisciplinarity, and patterns of research collaboration in nanoscience and nanotechnology," Scientometrics, Springer;Akadémiai Kiadó, vol. 59(3), pages 425-465, March.
    11. Keisuke Okamura, 2019. "Interdisciplinarity revisited: evidence for research impact and dynamism," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-9, December.
    12. Alan L. Porter & Alex S. Cohen & J. David Roessner & Marty Perreault, 2007. "Measuring researcher interdisciplinarity," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(1), pages 117-147, July.
    13. M. Bordons & M. A. Zulueta & F. Romero & S. Barrigón, 1999. "Measuring interdisciplinary collaboration within a university: The effects of the multidisciplinary research programme," Scientometrics, Springer;Akadémiai Kiadó, vol. 46(3), pages 383-398, November.
    14. Chiara Carusi & Giuseppe Bianchi, 2020. "A look at interdisciplinarity using bipartite scholar/journal networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 867-894, February.
    15. Shiji Chen & Clément Arsenault & Yves Gingras & Vincent Larivière, 2015. "Exploring the interdisciplinary evolution of a discipline: the case of Biochemistry and Molecular Biology," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1307-1323, February.
    16. Leydesdorff, Loet & Rafols, Ismael, 2011. "Indicators of the interdisciplinarity of journals: Diversity, centrality, and citations," Journal of Informetrics, Elsevier, vol. 5(1), pages 87-100.
    17. Andy Stirling, 2007. "A General Framework for Analysing Diversity in Science, Technology and Society," SPRU Working Paper Series 156, SPRU - Science Policy Research Unit, University of Sussex Business School.
    18. Mao, Jin & Liang, Zhentao & Cao, Yujie & Li, Gang, 2020. "Quantifying cross-disciplinary knowledge flow from the perspective of content: Introducing an approach based on knowledge memes," Journal of Informetrics, Elsevier, vol. 14(4).
    19. Fernanda Morillo & María Bordons & Isabel Gómez, 2001. "An approach to interdisciplinarity through bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 51(1), pages 203-222, April.
    20. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    21. Leydesdorff, Loet & Wagner, Caroline S. & Bornmann, Lutz, 2019. "Interdisciplinarity as diversity in citation patterns among journals: Rao-Stirling diversity, relative variety, and the Gini coefficient," Journal of Informetrics, Elsevier, vol. 13(1), pages 255-269.
    22. Alan L. Porter & Ismael Rafols, 2009. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 719-745, December.
    23. Qiuju Zhou & Ronald Rousseau & Liying Yang & Ting Yue & Guoliang Yang, 2012. "A general framework for describing diversity within systems and similarity between systems with applications in informetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 787-812, December.
    24. Jost, Lou, 2009. "Mismeasuring biological diversity: Response to Hoffmann and Hoffmann (2008)," Ecological Economics, Elsevier, vol. 68(4), pages 925-928, February.
    25. Lindell Bromham & Russell Dinnage & Xia Hua, 2016. "Interdisciplinary research has consistently lower funding success," Nature, Nature, vol. 534(7609), pages 684-687, June.
    26. Tibor Braun & András Schubert, 2003. "A quantitative view on the coming of age of interdisciplinarity in the sciences 1980-1999," Scientometrics, Springer;Akadémiai Kiadó, vol. 58(1), pages 183-189, September.
    27. Michael Gowanlock & Rich Gazan, 2013. "Assessing researcher interdisciplinarity: a case study of the University of Hawaii NASA Astrobiology Institute," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(1), pages 133-161, January.
    28. Sándor Soós & George Kampis, 2012. "Beyond the basemap of science: mapping multiple structures in research portfolios: evidence from Hungary," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 869-891, December.
    29. Loet Leydesdorff, 2007. "Betweenness centrality as an indicator of the interdisciplinarity of scientific journals," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(9), pages 1303-1319, July.
    30. Wagner, Caroline S. & Roessner, J. David & Bobb, Kamau & Klein, Julie Thompson & Boyack, Kevin W. & Keyton, Joann & Rafols, Ismael & Börner, Katy, 2011. "Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature," Journal of Informetrics, Elsevier, vol. 5(1), pages 14-26.
    31. Thomas W. Steele & Jeffrey C. Stier, 2000. "The impact of interdisciplinary research in the environmental sciences: a forestry case study," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 51(5), pages 476-484.
    32. Yu-Wei Chang & Mu-Hsuan Huang, 2012. "A study of the evolution of interdisciplinarity in library and information science: Using three bibliometric methods," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(1), pages 22-33, January.
    33. Jian Qin & F. W. Lancaster & Bryce Allen, 1997. "Types and levels of collaboration in interdisciplinary research in the sciences," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 48(10), pages 893-916, October.
    34. Yu‐Wei Chang & Mu‐Hsuan Huang, 2012. "A study of the evolution of interdisciplinarity in library and information science: Using three bibliometric methods," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(1), pages 22-33, January.
    35. Allan D. Pratt, 1977. "A measure of class concentration in bibliometrics," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 28(5), pages 285-292, September.
    36. Loet Leydesdorff & Félix Moya-Anegón & Vicente P. Guerrero-Bote, 2015. "Journal maps, interactive overlays, and the measurement of interdisciplinarity on the basis of Scopus data (1996–2012)," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(5), pages 1001-1016, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jingjing Ren & Fang Wang & Minglu Li, 2023. "Dynamics and characteristics of interdisciplinary research in scientific breakthroughs: case studies of Nobel-winning research in the past 120 years," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4383-4419, August.
    2. Giulio Giacomo Cantone, 2024. "How to measure interdisciplinary research? A systemic design for the model of measurement," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 4937-4982, August.
    3. Gabriele Sampagnaro, 2023. "Keyword occurrences and journal specialization," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5629-5645, October.
    4. Andrew Freiband & Katherine L. Dickin & Mitchell Glass & Michael A. Gore & Juan Hinestroza & Rebecca Nelson & Verity Platt & Noliwe Rooks & Aaron Sachs & Nathaniel Stern & Johannes Lehmann, 2022. "Undisciplining the university through shared purpose, practice, and place," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-10, December.
    5. Sampagnaro, Gabriele, 2024. "The disciplinarity of finance journals," International Review of Economics & Finance, Elsevier, vol. 93(PB), pages 374-388.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giulio Giacomo Cantone, 2024. "How to measure interdisciplinary research? A systemic design for the model of measurement," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(8), pages 4937-4982, August.
    2. Alfonso Ávila-Robinson & Cristian Mejia & Shintaro Sengoku, 2021. "Are bibliometric measures consistent with scientists’ perceptions? The case of interdisciplinarity in research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7477-7502, September.
    3. Chen, Shiji & Qiu, Junping & Arsenault, Clément & Larivière, Vincent, 2021. "Exploring the interdisciplinarity patterns of highly cited papers," Journal of Informetrics, Elsevier, vol. 15(1).
    4. Jingjing Ren & Fang Wang & Minglu Li, 2023. "Dynamics and characteristics of interdisciplinary research in scientific breakthroughs: case studies of Nobel-winning research in the past 120 years," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4383-4419, August.
    5. Xuefeng Wang & Zhinan Wang & Ying Huang & Yun Chen & Yi Zhang & Huichao Ren & Rongrong Li & Jinhui Pang, 2017. "Measuring interdisciplinarity of a research system: detecting distinction between publication categories and citation categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 2023-2039, June.
    6. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Zhang, Lin, 2018. "A comparison of two approaches for measuring interdisciplinary research output: The disciplinary diversity of authors vs the disciplinary diversity of the reference list," Journal of Informetrics, Elsevier, vol. 12(4), pages 1182-1193.
    7. Chen, Shiji & Arsenault, Clément & Larivière, Vincent, 2015. "Are top-cited papers more interdisciplinary?," Journal of Informetrics, Elsevier, vol. 9(4), pages 1034-1046.
    8. Shiji Chen & Yanhui Song & Fei Shu & Vincent Larivière, 2022. "Interdisciplinarity and impact: the effects of the citation time window," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2621-2642, May.
    9. Hongyu Zhou & Raf Guns & Tim C. E. Engels, 2022. "Are social sciences becoming more interdisciplinary? Evidence from publications 1960–2014," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(9), pages 1201-1221, September.
    10. Lin Zhang & Beibei Sun & Zaida Chinchilla-Rodríguez & Lixin Chen & Ying Huang, 2018. "Interdisciplinarity and collaboration: on the relationship between disciplinary diversity in departmental affiliations and reference lists," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 271-291, October.
    11. Shunshun Shi & Wenyu Zhang & Shuai Zhang & Jie Chen, 2018. "Does prestige dimension influence the interdisciplinary performance of scientific entities in knowledge flow? Evidence from the e-government field," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(2), pages 1237-1264, November.
    12. Chiara Carusi & Giuseppe Bianchi, 2020. "A look at interdisciplinarity using bipartite scholar/journal networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 867-894, February.
    13. Shengli Deng & Sudi Xia, 2020. "Mapping the interdisciplinarity in information behavior research: a quantitative study using diversity measure and co-occurrence analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 489-513, July.
    14. Lina Xu & Steven Dellaportas & Jin Wang, 2022. "A study of interdisciplinary accounting research: analysing the diversity of cited references," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 62(2), pages 2131-2162, June.
    15. Qing Ke, 2023. "Interdisciplinary research and technological impact: evidence from biomedicine," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2035-2077, April.
    16. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    17. Lina Xu & Steven Dellaportas & Zhiqiang Yang & Jin Wang, 2023. "More on the relationship between interdisciplinary accounting research and citation impact," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 63(4), pages 4779-4803, December.
    18. Ricardo Arencibia-Jorge & Rosa Lidia Vega-Almeida & José Luis Jiménez-Andrade & Humberto Carrillo-Calvet, 2022. "Evolutionary stages and multidisciplinary nature of artificial intelligence research," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5139-5158, September.
    19. Tracy Klarenbeek & Nelius Boshoff, 2018. "Measuring multidisciplinary health research at South African universities: a comparative analysis based on co-authorships and journal subject categories," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1461-1485, September.
    20. Keisuke Okamura, 2019. "Interdisciplinarity revisited: evidence for research impact and dynamism," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-9, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:127:y:2022:i:12:d:10.1007_s11192-022-04310-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.