IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v122y2020i2d10.1007_s11192-019-03333-3.html
   My bibliography  Save this article

In search of better science: on the epistemic costs of systematic reviews and the need for a pluralistic stance to literature search

Author

Listed:
  • Andrea Polonioli

    (Coveo Solutions Inc)

Abstract

This paper reviews the current status of academic search engines and emerging trends in scientific information retrieval and argues for two key claims. First, since systematic searches rely on the widespread use of academic search engines and the latter are generally not powered by cutting-edge Artificial Intelligence (AI) and not well-positioned to further the goals of findability and discoverability, there are some non-trivial epistemic costs associated with the tradition of systematic search. Second, while narrative reviews are typically criticized because of their lack of transparency, accountability, and reproducibility, they do deserve a place in scientific research. Specifically, once narrative reviews are properly understood as enabled by modern tools such as non-academic search engines, AI-powered recommender systems and academic social networks, it is possible to appreciate how these can indeed further the goal of literature discoverability. The upshot of this piece is that there are multiple goals and trade-offs involved in the process of scientific document search and that we should acknowledge virtues and limitations of different approaches to information retrieval and be prepared to welcome their combined use.

Suggested Citation

  • Andrea Polonioli, 2020. "In search of better science: on the epistemic costs of systematic reviews and the need for a pluralistic stance to literature search," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1267-1274, February.
    • Handle: RePEc:spr:scient:v:122:y:2020:i:2:d:10.1007_s11192-019-03333-3
    DOI: 10.1007/s11192-019-03333-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-019-03333-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-019-03333-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lutz Bornmann & Rüdiger Mutz, 2015. "Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2215-2222, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andrea Polonioli, 2021. "The ethics of scientific recommender systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1841-1848, February.
    2. Gianpaolo Abatecola & Alberto Surace, 2020. "Discussing the Use of Complexity Theory in Engineering Management: Implications for Sustainability," Sustainability, MDPI, vol. 12(24), pages 1-24, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ramona Weinrich, 2019. "Opportunities for the Adoption of Health-Based Sustainable Dietary Patterns: A Review on Consumer Research of Meat Substitutes," Sustainability, MDPI, vol. 11(15), pages 1-15, July.
    2. Piers Steel & Sjoerd Beugelsdijk & Herman Aguinis, 2021. "The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 52(1), pages 23-44, February.
    3. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    4. Augusteijn, Hilde Elisabeth Maria & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2021. "Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias," OSF Preprints avkgj, Center for Open Science.
    5. Ruhua Huang & Yuting Huang & Fan Qi & Leyi Shi & Baiyang Li & Wei Yu, 2022. "Exploring the characteristics of special issues: distribution, topicality, and citation impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5233-5256, September.
    6. Neal R. Haddaway & Max W. Callaghan & Alexandra M. Collins & William F. Lamb & Jan C. Minx & James Thomas & Denny John, 2020. "On the use of computer‐assistance to facilitate systematic mapping," Campbell Systematic Reviews, John Wiley & Sons, vol. 16(4), December.
    7. Vincent Raoult, 2020. "How Many Papers Should Scientists Be Reviewing? An Analysis Using Verified Peer Review Reports," Publications, MDPI, vol. 8(1), pages 1-9, January.
    8. Eloy López-Meneses & Esteban Vázquez-Cano & Mariana-Daniela González-Zamar & Emilio Abad-Segura, 2020. "Socioeconomic Effects in Cyberbullying: Global Research Trends in the Educational Context," IJERPH, MDPI, vol. 17(12), pages 1-31, June.
    9. Jinseok Kim & Jinmo Kim & Jason Owen-Smith, 2019. "Generating automatically labeled data for author name disambiguation: an iterative clustering method," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 253-280, January.
    10. Soo Jeung Lee & Christian Schneijderberg & Yangson Kim & Isabel Steinhardt, 2021. "Have Academics’ Citation Patterns Changed in Response to the Rise of World University Rankings? A Test Using First-Citation Speeds," Sustainability, MDPI, vol. 13(17), pages 1-19, August.
    11. Lutz Bornmann & Robin Haunschild & Rüdiger Mutz, 2021. "Growth rates of modern science: a latent piecewise growth curve approach to model publication numbers from established and new literature databases," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-15, December.
    12. Yu Zhang & Min Wang & Morteza Saberi & Elizabeth Chang, 2022. "Analysing academic paper ranking algorithms using test data and benchmarks: an investigation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(7), pages 4045-4074, July.
    13. Edré Moreira & Wagner Meira & Marcos André Gonçalves & Alberto H. F. Laender, 2023. "The rise of hyperprolific authors in computer science: characterization and implications," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2945-2974, May.
    14. Lanu Kim & Jason H. Portenoy & Jevin D. West & Katherine W. Stovel, 2020. "Scientific journals still matter in the era of academic search engines and preprint archives," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(10), pages 1218-1226, October.
    15. Ana Teresa Santos & Sandro Mendonça, 2022. "Do papers (really) match journals’ “aims and scope”? A computational assessment of innovation studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(12), pages 7449-7470, December.
    16. Jensen, Scott & Liu, Xiaozhong & Yu, Yingying & Milojevic, Staša, 2016. "Generation of topic evolution trees from heterogeneous bibliographic networks," Journal of Informetrics, Elsevier, vol. 10(2), pages 606-621.
    17. Bornmann, Lutz & Haunschild, Robin, 2022. "Empirical analysis of recent temporal dynamics of research fields: Annual publications in chemistry and related areas as an example," Journal of Informetrics, Elsevier, vol. 16(2).
    18. Carlo Galli & Stefano Guizzardi, 2021. "The Effect of Article Characteristics on Citation Number in a Diachronic Dataset of the Biomedical Literature on Chronic Inflammation: An Analysis by Ensemble Machines," Publications, MDPI, vol. 9(2), pages 1-11, April.
    19. Hong Shi & Mengmeng Cheng & Yi Feng & Chenghui Qiu & Caiyue Song & Nenglin Yuan & Chuanzhi Kang & Kaijie Yang & Jie Yuan & Yonghao Li, 2023. "Thermal Management Techniques for Lithium-Ion Batteries Based on Phase Change Materials: A Systematic Review and Prospective Recommendations," Energies, MDPI, vol. 16(2), pages 1-23, January.
    20. Andrej Kastrin & Dimitar Hristovski, 2021. "Scientometric analysis and knowledge mapping of literature-based discovery (1986–2020)," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1415-1451, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:122:y:2020:i:2:d:10.1007_s11192-019-03333-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.