IDEAS home Printed from https://ideas.repec.org/a/oup/rseval/v31y2022i3p326-343..html
   My bibliography  Save this article

Pathways for assessing interdisciplinarity: A systematic review

Author

Listed:
  • Bethany K Laursen
  • Nicole Motzer
  • Kelly J Anderson

Abstract

In many sectors and in many respects, interdisciplinarity has made progress in recent decades, but less so when it comes to evaluating interdisciplinary work. Challenges remain, especially regarding what counts as ‘good’ interdisciplinarity. While previous reviews describe broad trends contributing to these challenges, high-level observations are less suited to: (1) pinpointing underlying sources of persistent evaluative issues, (2) capturing less frequent counterexamples and emerging trends, and (3) providing practical guidance for moving the field forward. This article presents a novel approach for excavating and analyzing assessment designs within and across a wide range of published studies. It is the first study to structure its review of interdisciplinary evaluation around the basics of evaluation theory, creating a simplified model of assessment design choices called an assessment pathway. It further stands alone as an extensive systematic review of the interdisciplinary literature uniquely placed within the context of evaluation, encompassing 20 years, 142 studies, 1,006 assessment pathways, and 19,114 data points. Results show that while interdisciplinary assessments to date excel at variation, only 12% of identified pathways contain the minimum elements required to reach an evaluative judgment. Increased attention to evaluation theory can catalyze progress and provide structure and shared language to a highly varied domain. Readers can visually explore the full dataset online and download it from Harvard’s Dataverse.

Suggested Citation

  • Bethany K Laursen & Nicole Motzer & Kelly J Anderson, 2022. "Pathways for assessing interdisciplinarity: A systematic review," Research Evaluation, Oxford University Press, vol. 31(3), pages 326-343.
  • Handle: RePEc:oup:rseval:v:31:y:2022:i:3:p:326-343.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/reseval/rvac013
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Arsev U. Aydinoglu & Suzie Allard & Chad Mitchell, 2016. "Measuring diversity in disciplinary collaboration in research teams: An ecological perspective," Research Evaluation, Oxford University Press, vol. 25(1), pages 18-36.
    2. Jennifer Leigh & Nicole Brown, 2021. "Researcher experiences in practice-based interdisciplinary research [Imagining Autism: Feasibility and Impact of a Drama-Based Intervention on the Social Communicative and Imaginative Behaviour of ," Research Evaluation, Oxford University Press, vol. 30(4), pages 421-430.
    3. Leahey, Erin & Barringer, Sondra N., 2020. "Universities’ commitment to interdisciplinary research: To what end?," Research Policy, Elsevier, vol. 49(2).
    4. Huutoniemi, Katri & Klein, Julie Thompson & Bruun, Henrik & Hukkinen, Janne, 2010. "Analyzing interdisciplinarity: Typology and indicators," Research Policy, Elsevier, vol. 39(1), pages 79-88, February.
    5. Bammer, Gabriele, 2008. "Enhancing research collaborations: Three key management challenges," Research Policy, Elsevier, vol. 37(5), pages 875-887, June.
    6. repec:pal:palcom:v:2016:y:2016:i:palcomms201655:p:16055- is not listed on IDEAS
    7. Lin Zhang & Beibei Sun & Lidan Jiang & Ying Huang, 2021. "On the relationship between interdisciplinarity and impact: Distinct effects on academic and broader impact [A Comparison of Two Approaches for Measuring Interdisciplinary Research Output: The Disc," Research Evaluation, Oxford University Press, vol. 30(3), pages 256-268.
    8. Irwin Feller, 2006. "Multiple actors, multiple settings, multiple criteria: issues in assessing interdisciplinary research," Research Evaluation, Oxford University Press, vol. 15(1), pages 5-15, April.
    9. Keisuke Okamura, 2019. "Interdisciplinarity revisited: evidence for research impact and dynamism," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-9, December.
    10. Rick Rylance, 2015. "Grant giving: Global funders to focus on interdisciplinarity," Nature, Nature, vol. 525(7569), pages 313-315, September.
    11. Grit Laudel & Gloria Origgi, 2006. "Introduction to a special issue on the assessment of interdisciplinary research," Research Evaluation, Oxford University Press, vol. 15(1), pages 2-4, April.
    12. Veronica Boix Mansilla & Elizabeth Dawes Duraisingh & Christopher R. Wolfe & Carolyn Haynes, 2009. "Targeted Assessment Rubric: An Empirically Grounded Rubric for Interdisciplinary Writing," The Journal of Higher Education, Taylor & Francis Journals, vol. 80(3), pages 334-353, May.
    13. Lungeanu, Alina & Huang, Yun & Contractor, Noshir S., 2014. "Understanding the assembly of interdisciplinary teams and its impact on performance," Journal of Informetrics, Elsevier, vol. 8(1), pages 59-70.
    14. Julie Thompson Klein, 2006. "Afterword: the emergent literature on interdisciplinary and transdisciplinary research evaluation," Research Evaluation, Oxford University Press, vol. 15(1), pages 75-80, April.
    15. Jurian Edelenbos & Nanny Bressers & Lieselot Vandenbussche, 2017. "Evolution of interdisciplinary collaboration: what are stimulating conditions?," Science and Public Policy, Oxford University Press, vol. 44(4), pages 451-463.
    16. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Zhang, Lin, 2018. "A comparison of two approaches for measuring interdisciplinary research output: The disciplinary diversity of authors vs the disciplinary diversity of the reference list," Journal of Informetrics, Elsevier, vol. 12(4), pages 1182-1193.
    17. Sondra N. Barringer & Erin Leahey & Karina Salazar, 2020. "What Catalyzes Research Universities to Commit to Interdisciplinary Research?," Research in Higher Education, Springer;Association for Institutional Research, vol. 61(6), pages 679-705, September.
    18. Alan L. Porter & Ismael Rafols, 2009. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 719-745, December.
    19. David Roessner & Alan L. Porter & Nancy J. Nersessian & Stephen Carley, 2013. "Validating indicators of interdisciplinarity: linking bibliometric measures to studies of engineering research labs," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 439-468, February.
    20. Veronica Boix Mansilla, 2006. "Assessing expert interdisciplinary work at the frontier: an empirical exploration," Research Evaluation, Oxford University Press, vol. 15(1), pages 17-29, April.
    21. Wagner, Caroline S. & Roessner, J. David & Bobb, Kamau & Klein, Julie Thompson & Boyack, Kevin W. & Keyton, Joann & Rafols, Ismael & Börner, Katy, 2011. "Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature," Journal of Informetrics, Elsevier, vol. 5(1), pages 14-26.
    22. Alan L Porter & David J Roessner & Anne E Heberger, 2008. "How interdisciplinary is a given body of research?," Research Evaluation, Oxford University Press, vol. 17(4), pages 273-282, December.
    23. Brian M. Belcher & Katherine E. Rasmussen & Matthew R. Kemshaw & Deborah A. Zornes, 2016. "Defining and assessing research quality in a transdisciplinary context," Research Evaluation, Oxford University Press, vol. 25(1), pages 1-17.
    24. Tom McLeish & Veronica Strang, 2016. "Evaluating interdisciplinary research: the elephant in the peer-reviewers’ room," Palgrave Communications, Palgrave Macmillan, vol. 2(1), pages 1-8, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andrea Bonaccorsi & Nicola Melluso & Francesco Alessandro Massucci, 2022. "Exploring the antecedents of interdisciplinarity at the European Research Council: a topic modeling approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 6961-6991, December.
    2. Francesco Giovanni Avallone & Alberto Quagli & Paola Ramassa, 2022. "Interdisciplinary research by accounting scholars: An exploratory study," FINANCIAL REPORTING, FrancoAngeli Editore, vol. 2022(2), pages 5-34.
    3. Meijun Liu & Sijie Yang & Yi Bu & Ning Zhang, 2023. "Female early-career scientists have conducted less interdisciplinary research in the past six decades: evidence from doctoral theses," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-16, December.
    4. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    5. Fontana, Magda & Iori, Martina & Leone Sciabolazza, Valerio & Souza, Daniel, 2022. "The interdisciplinarity dilemma: Public versus private interests," Research Policy, Elsevier, vol. 51(7).
    6. Alfonso Ávila-Robinson & Cristian Mejia & Shintaro Sengoku, 2021. "Are bibliometric measures consistent with scientists’ perceptions? The case of interdisciplinarity in research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7477-7502, September.
    7. Lin Zhang & Beibei Sun & Zaida Chinchilla-Rodríguez & Lixin Chen & Ying Huang, 2018. "Interdisciplinarity and collaboration: on the relationship between disciplinary diversity in departmental affiliations and reference lists," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 271-291, October.
    8. Cian O’Donovan & Aleksandra (Ola) Michalec & Joshua R Moon, 2022. "Capabilities for transdisciplinary research," Research Evaluation, Oxford University Press, vol. 31(1), pages 145-158.
    9. Alexis-Michel Mugabushaka & Anthi Kyriakou & Theo Papazoglou, 2016. "Bibliometric indicators of interdisciplinarity: the potential of the Leinster–Cobbold diversity indices to study disciplinary diversity," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 593-607, May.
    10. Lorenzo Cassi & Wilfriedo Mescheba & Élisabeth Turckheim, 2014. "How to evaluate the degree of interdisciplinarity of an institution?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1871-1895, December.
    11. Lina Xu & Steven Dellaportas & Jin Wang, 2022. "A study of interdisciplinary accounting research: analysing the diversity of cited references," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 62(2), pages 2131-2162, June.
    12. Oviedo-García, M. Ángeles, 2016. "Tourism research quality: Reviewing and assessing interdisciplinarity," Tourism Management, Elsevier, vol. 52(C), pages 586-592.
    13. Jingjing Ren & Fang Wang & Minglu Li, 2023. "Dynamics and characteristics of interdisciplinary research in scientific breakthroughs: case studies of Nobel-winning research in the past 120 years," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4383-4419, August.
    14. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Zhang, Lin, 2018. "A comparison of two approaches for measuring interdisciplinary research output: The disciplinary diversity of authors vs the disciplinary diversity of the reference list," Journal of Informetrics, Elsevier, vol. 12(4), pages 1182-1193.
    15. Casey Helgeson & Robert E. Nicholas & Klaus Keller & Chris E. Forest & Nancy Tuana, 2022. "Attention to values helps shape convergence research," Climatic Change, Springer, vol. 170(1), pages 1-19, January.
    16. Sander Zwanenburg & Maryam Nakhoda & Peter Whigham, 2022. "Toward greater consistency and validity in measuring interdisciplinarity: a systematic and conceptual evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7769-7788, December.
    17. Arnold, Austin & Cafer, Anne & Green, John & Haines, Seena & Mann, Georgianna & Rosenthal, Meagen, 2021. "“Perspective: Promoting and fostering multidisciplinary research in universities”," Research Policy, Elsevier, vol. 50(9).
    18. Koppman, Sharon & Leahey, Erin, 2019. "Who moves to the methodological edge? Factors that encourage scientists to use unconventional methods," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    19. Carr, Gemma & Loucks, Daniel P. & Blöschl, Günter, 2018. "Gaining insight into interdisciplinary research and education programmes: A framework for evaluation," Research Policy, Elsevier, vol. 47(1), pages 35-48.
    20. Stephen Carley & Alan L. Porter, 2012. "A forward diversity index," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 407-427, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:rseval:v:31:y:2022:i:3:p:326-343.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/rev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.