IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i12p2135-d842584.html
   My bibliography  Save this article

Analytics Methods to Understand Information Retrieval Effectiveness—A Survey

Author

Listed:
  • Josiane Mothe

    (INSPE, IRIT UMR5505 CNRS, Université Toulouse Jean-Jaurès, 118 Rte de Narbonne, F-31400 Toulouse, France)

Abstract

Information retrieval aims to retrieve the documents that answer users’ queries. A typical search process consists of different phases for which a variety of components have been defined in the literature; each one having a set of hyper-parameters to tune. Different studies focused on how and how much the components and their hyper-parameters affect the system performance in terms of effectiveness, others on the query factor. The aim of these studies is to better understand information retrieval system effectiveness. This paper reviews the literature of this domain. It depicts how data analytics has been used in IR to gain a better understanding of system effectiveness. This review concludes that we lack a full understanding of system effectiveness related to the context which the system is in, though it has been possible to adapt the query processing to some contexts successfully. This review also concludes that, even if it is possible to distinguish effective from non-effective systems for a query set, neither the system component analysis nor the query features analysis were successful in explaining when and why a particular system fails on a particular query.

Suggested Citation

  • Josiane Mothe, 2022. "Analytics Methods to Understand Information Retrieval Effectiveness—A Survey," Mathematics, MDPI, vol. 10(12), pages 1-25, June.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:12:p:2135-:d:842584
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/12/2135/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/12/2135/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Nicola Ferro & Gianmaria Silvello, 2018. "Toward an anatomy of IR system component performances," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(2), pages 187-200, February.
    2. S. E. Robertson & K. Sparck Jones, 1976. "Relevance weighting of search terms," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 27(3), pages 129-146, May.
    3. Falk Scholer & Hugh E. Williams & Andrew Turpin, 2004. "Query association surrogates for Web search," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 55(7), pages 637-650, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Edward Kai Fung Dang & Robert Wing Pong Luk & James Allan, 2022. "A retrieval model family based on the probability ranking principle for ad hoc retrieval," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(8), pages 1140-1154, August.
    2. Victoria Yoon & Bonnie Rubenstein Montano & Teresa Wilson & Stuart Lowry & Jay Liebowitz, 2004. "Natural language interface for multi‐agent contracting system (MACS)," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 12(3), pages 153-165, July.
    3. Lynda Tamine & Cécile Chouquet & Thomas Palmer, 2015. "Analysis of biomedical and health queries: Lessons learned from TREC," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(12), pages 2626-2642, December.
    4. Adrian-Gabriel Chifu & Sébastien Fournier, 2023. "Sentiment Difficulty in Aspect-Based Sentiment Analysis," Mathematics, MDPI, vol. 11(22), pages 1-33, November.
    5. Jean-Charles Lamirel & Claire Francois & Shadi Al Shehabi & Martial Hoffmann, 2004. "New classification quality estimators for analysis of documentary information: Application to patent analysis and web mapping," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(3), pages 445-562, August.
    6. Jiwon Yu & Jong-Gyu Hwang & Jumi Hwang & Sung Chan Jun & Sumin Kang & Chulung Lee & Hyundong Kim, 2020. "Identification of Vacant and Emerging Technologies in Smart Mobility Through the GTM-Based Patent Map Development," Sustainability, MDPI, vol. 12(22), pages 1-22, November.
    7. Huan Wang & Jian Li & Jiapeng Wang, 2023. "Retrieving Chinese Questions and Answers Based on Deep-Learning Algorithm," Mathematics, MDPI, vol. 11(18), pages 1-18, September.
    8. Jerry Ellig & Patrick A. McLaughlin, 2016. "The Regulatory Determinants of Railroad Safety," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 49(2), pages 371-398, September.
    9. Alexandra Dumitrescu & Simone Santini, 2021. "Full coverage of a reader's interests in context‐based information filtering," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(8), pages 1011-1027, August.
    10. Jean-Charles Lamirel & Shadi Al Shehabi & Claire Francois & Xavier Polanco, 2004. "Using a compound approach based on elaborated neural network for Webometrics: An example issued from the EICSTES project," Scientometrics, Springer;Akadémiai Kiadó, vol. 61(3), pages 427-441, November.
    11. Jean-Charles Lamirel, 2012. "A new approach for automatizing the analysis of research topics dynamics: application to optoelectronics research," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(1), pages 151-166, October.
    12. Kevin W Boyack & David Newman & Russell J Duhon & Richard Klavans & Michael Patek & Joseph R Biberstine & Bob Schijvenaars & André Skupin & Nianli Ma & Katy Börner, 2011. "Clustering More than Two Million Biomedical Publications: Comparing the Accuracies of Nine Text-Based Similarity Approaches," PLOS ONE, Public Library of Science, vol. 6(3), pages 1-11, March.
    13. Georgia Warren-Myers & Monique Schmidt, 2023. "The Evolving Nature (or Not) of Sustainability Communications in New Home Building in Australia," Sustainability, MDPI, vol. 15(19), pages 1-20, September.
    14. Guozhong Feng & Baiguo An & Fengqin Yang & Han Wang & Libiao Zhang, 2017. "Relevance popularity: A term event model based feature selection scheme for text classification," PLOS ONE, Public Library of Science, vol. 12(4), pages 1-15, April.
    15. Yunlong Ma & Hongfei Lin, 2014. "A Multiple Relevance Feedback Strategy with Positive and Negative Models," PLOS ONE, Public Library of Science, vol. 9(8), pages 1-10, August.
    16. Müge Akbulut & Yaşar Tonta & Howard D. White, 2020. "Related records retrieval and pennant retrieval: an exploratory case study," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 957-987, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:12:p:2135-:d:842584. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.