IDEAS home Printed from https://ideas.repec.org/a/bla/jinfst/v74y2023i8p1010-1025.html
   My bibliography  Save this article

Bibliometric‐enhanced legal information retrieval: Combining usage and citations as flavors of impact relevance

Author

Listed:
  • Gineke Wiggers
  • Suzan Verberne
  • Wouter van Loon
  • Gerrit‐Jan Zwenne

Abstract

Bibliometric‐enhanced information retrieval uses bibliometrics (e.g., citations) to improve ranking algorithms. Using a data‐driven approach, this article describes the development of a bibliometric‐enhanced ranking algorithm for legal information retrieval, and the evaluation thereof. We statistically analyze the correlation between usage of documents and citations over time, using data from a commercial legal search engine. We then propose a bibliometric boost function that combines usage of documents with citation counts. The core of this function is an impact variable based on usage and citations that increases in influence as citations and usage counts become more reliable over time. We evaluate our ranking function by comparing search sessions before and after the introduction of the new ranking in the search engine. Using a cost model applied to 129,571 sessions before and 143,864 sessions after the intervention, we show that our bibliometric‐enhanced ranking algorithm reduces the time of a search session of legal professionals by 2 to 3% on average for use cases other than known‐item retrieval or updating behavior. Given the high hourly tariff of legal professionals and the limited time they can spend on research, this is expected to lead to increased efficiency, especially for users with extremely long search sessions.

Suggested Citation

  • Gineke Wiggers & Suzan Verberne & Wouter van Loon & Gerrit‐Jan Zwenne, 2023. "Bibliometric‐enhanced legal information retrieval: Combining usage and citations as flavors of impact relevance," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(8), pages 1010-1025, August.
    • Handle: RePEc:bla:jinfst:v:74:y:2023:i:8:p:1010-1025
    DOI: 10.1002/asi.24799
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asi.24799
    Download Restriction: no
    File URL: https://libkey.io/10.1002/asi.24799?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Tefko Saracevic, 2007. "Relevance: A review of the literature and a framework for thinking on the notion in information science. Part III: Behavior and effects of relevance," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(13), pages 2126-2144, November.
    2. Alesia Zuccala & Roberto Cornacchia, 2016. "Data matching, integration, and interoperability for a metric assessment of monographs," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 465-484, July.
    3. Tefko Saracevic, 1975. "RELEVANCE: A review of and a framework for the thinking on the notion in information science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 26(6), pages 321-343, November.
    4. Tefko Saracevic, 2007. "Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(13), pages 1915-1933, November.
    5. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    6. Ashraf Maleki, 2022. "OCLC library holdings: assessing availability of academic books in libraries in print and electronic compared to citations and altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 991-1020, February.
    7. Tim Brody & Stevan Harnad & Leslie Carr, 2006. "Earlier Web usage statistics as predictors of later citation impact," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(8), pages 1060-1072, June.
    8. William Hersh, 1994. "Relevance and retrieval evaluation: Perspectives from medicine," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 45(3), pages 201-206, April.
    9. Michael D. Cooper & Hui‐Min Chen, 2001. "Predicting the relevance of a library catalog search," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 52(10), pages 813-827.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wayne de Fremery & Michael K. Buckland, 2022. "Context, relevance, and labor," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(9), pages 1268-1278, September.
    2. Anton van der Vegt & Guido Zuccon & Bevan Koopman, 2021. "Do better search engines really equate to better clinical decisions? If not, why not?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(2), pages 141-155, February.
    3. Roland Grad & Pierre Pluye & Vera Granikov & Janique Johnson‐Lafleur & Michael Shulha & Soumya Bindiganavile Sridhar & Jonathan L. Moscovici & Gillian Bartlett & Alain C. Vandal & Bernard Marlow & Lor, 2011. "Physicians' assessment of the value of clinical information: Operationalization of a theoretical model," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(10), pages 1884-1891, October.
    4. Xiaoli Huang & Dagobert Soergel, 2013. "Relevance: An improved framework for explicating the notion," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 18-35, January.
    5. Barbara McGillivray & Paola Marongiu & Nilo Pedrazzini & Marton Ribary & Mandy Wigdorowitz & Eleonora Zordan, 2022. "Deep Impact: A Study on the Impact of Data Papers and Datasets in the Humanities and Social Sciences," Publications, MDPI, vol. 10(4), pages 1-40, October.
    6. Brito, Ana C.M. & Silva, Filipi N. & de Arruda, Henrique F. & Comin, Cesar H. & Amancio, Diego R. & Costa, Luciano da F., 2021. "Classification of abrupt changes along viewing profiles of scientific articles," Journal of Informetrics, Elsevier, vol. 15(2).
    7. Julianne Sansa-Otim & Mary Nsabagwa & Andrew Mwesigwa & Becky Faith & Mojisola Owoseni & Olayinka Osuolale & Daudi Mboma & Ben Khemis & Peter Albino & Samuel Owusu Ansah & Maureen Abla Ahiataku & Vict, 2022. "An Assessment of the Effectiveness of Weather Information Dissemination among Farmers and Policy Makers," Sustainability, MDPI, vol. 14(7), pages 1-20, March.
    8. Maram Hasanain & Tamer Elsayed, 2022. "Studying effectiveness of Web search for fact checking," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(5), pages 738-751, May.
    9. Rongying Zhao & Mingkun Wei, 2017. "Academic impact evaluation of Wechat in view of social media perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1777-1791, September.
    10. Sergio Copiello, 2020. "The alleged citation advantage of video abstracts may be a matter of self-citations and self-selection bias. Comment on “The impact of video abstract on citation counts” by Zong et al," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 751-757, January.
    11. Zhang, Lei & Kopak, Rick & Freund, Luanne & Rasmussen, Edie, 2011. "Making functional units functional: The role of rhetorical structure in use of scholarly journal articles," International Journal of Information Management, Elsevier, vol. 31(1), pages 21-29.
    12. Howard D. White, 2015. "Co-cited author retrieval and relevance theory: examples from the humanities," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2275-2299, March.
    13. Frans van der Sluis & Egon L. van den Broek, 2023. "Feedback beyond accuracy: Using eye‐tracking to detect comprehensibility and interest during reading," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 3-16, January.
    14. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    15. Cristina López-Duarte & Jane F. Maley & Marta M. Vidal-Suárez, 2021. "Main challenges to international student mobility in the European arena," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 8957-8980, November.
    16. Barbara McGillivray & Mathias Astell, 2019. "The relationship between usage and citations in an open access mega-journal," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 817-838, November.
    17. Ying Guo & Xiantao Xiao, 2022. "Author-level altmetrics for the evaluation of Chinese scholars," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 973-990, February.
    18. Daniela De Filippo & Fernanda Morillo & Borja González-Albo, 2023. "Measuring the Impact and Influence of Scientific Activity in the Humanities and Social Sciences," Publications, MDPI, vol. 11(2), pages 1-17, June.
    19. Jianhua Hou & Xiucai Yang & Yang Zhang, 2023. "The effect of social media knowledge cascade: an analysis of scientific papers diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5169-5195, September.
    20. Isidro F. Aguillo, 2020. "Altmetrics of the Open Access Institutional Repositories: a webometrics approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(3), pages 1181-1192, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jinfst:v:74:y:2023:i:8:p:1010-1025. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.asis.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.