IDEAS home Printed from https://ideas.repec.org/a/bla/jinfst/v68y2017i12p2743-2754.html
   My bibliography  Save this article

Understanding an enriched multidimensional user relevance model by analyzing query logs

Author

Listed:
  • Jingfei Li
  • Peng Zhang
  • Dawei Song
  • Yue Wu

Abstract

Modeling multidimensional relevance in information retrieval (IR) has attracted much attention in recent years. However, most existing studies are conducted through relatively small‐scale user studies, which may not reflect a real‐world and natural search scenario. In this article, we propose to study the multidimensional user relevance model (MURM) on large scale query logs, which record users' various search behaviors (e.g., query reformulations, clicks and dwelling time, etc.) in natural search settings. We advance an existing MURM model (including five dimensions: topicality, novelty, reliability, understandability, and scope) by providing two additional dimensions, that is, interest and habit. The two new dimensions represent personalized relevance judgment on retrieved documents. Further, for each dimension in the enriched MURM model, a set of computable features are formulated. By conducting extensive document ranking experiments on Bing's query logs and TREC session Track data, we systematically investigated the impact of each dimension on retrieval performance and gained a series of insightful findings which may bring benefits for the design of future IR systems.

Suggested Citation

  • Jingfei Li & Peng Zhang & Dawei Song & Yue Wu, 2017. "Understanding an enriched multidimensional user relevance model by analyzing query logs," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(12), pages 2743-2754, December.
  • Handle: RePEc:bla:jinfst:v:68:y:2017:i:12:p:2743-2754
    DOI: 10.1002/asi.23868
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asi.23868
    Download Restriction: no

    File URL: https://libkey.io/10.1002/asi.23868?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Carol L. Barry, 1998. "Document representations and clues to document relevance," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 49(14), pages 1293-1303.
    2. Anastasios Tombros & Ian Ruthven & Joemon M. Jose, 2005. "How users assess Web pages for information seeking," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 56(4), pages 327-344, February.
    3. Tefko Saracevic, 1975. "RELEVANCE: A review of and a framework for the thinking on the notion in information science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 26(6), pages 321-343, November.
    4. Yunjie (Calvin) Xu & Zhiwei Chen, 2006. "Relevance judgment: What do information users consider beyond topicality?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(7), pages 961-973, May.
    5. Yunjie Xu & Hainan Yin, 2008. "Novelty and topicality in interactive information retrieval," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(2), pages 201-215, January.
    6. John J. Regazzi, 1988. "Performance measures for information retrieval systems—an experimental approach," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 39(4), pages 235-251, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Antonio Maria Rinaldi & Cristiano Russo & Cristian Tommasino, 2020. "A Knowledge-Driven Multimedia Retrieval System Based on Semantics and Deep Features," Future Internet, MDPI, vol. 12(11), pages 1-20, October.
    2. Xiaoli Huang & Dagobert Soergel, 2013. "Relevance: An improved framework for explicating the notion," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 18-35, January.
    3. Alexandra Dumitrescu & Simone Santini, 2021. "Full coverage of a reader's interests in context‐based information filtering," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(8), pages 1011-1027, August.
    4. Nathalie Demoulin & Kristof Coussement, 2018. "Acceptance of text-mining systems: The signaling role of information quality," Post-Print hal-02111772, HAL.
    5. Anton van der Vegt & Guido Zuccon & Bevan Koopman, 2021. "Do better search engines really equate to better clinical decisions? If not, why not?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(2), pages 141-155, February.
    6. Malcolm Dow & Peter Willett & Roderick McDonald & Belver Griffith & Michael Greenacre & Peter Bryant & Daniel Wartenberg & Ove Frank, 1987. "Book reviews," Journal of Classification, Springer;The Classification Society, vol. 4(2), pages 245-278, September.
    7. repec:jtr:journl:v:4:y:2012:i:1:p:12-37 is not listed on IDEAS
    8. Xiuping Zhang & Jaewon Choi, 2022. "The Importance of Social Influencer-Generated Contents for User Cognition and Emotional Attachment: An Information Relevance Perspective," Sustainability, MDPI, vol. 14(11), pages 1-18, May.
    9. Howard D. White, 2010. "Some new tests of relevance theory in information science," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(3), pages 653-667, June.
    10. Warinrampai Rungruangjit & Kitti Charoenpornpanichkul, 2022. "Building Stronger Brand Evangelism for Sustainable Marketing through Micro-Influencer-Generated Content on Instagram in the Fashion Industry," Sustainability, MDPI, vol. 14(23), pages 1-27, November.
    11. Dietmar Wolfram, 2015. "The symbiotic relationship between information retrieval and informetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2201-2214, March.
    12. Yan Chen & Grace Jeon & Yong-Mi Kim, 2014. "A day without a search engine: an experimental study of online and offline searches," Experimental Economics, Springer;Economic Science Association, vol. 17(4), pages 512-536, December.
    13. Yong‐Mi Kim, 2010. "The adoption of university library Web site resources: A multigroup analysis," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(5), pages 978-993, May.
    14. Anton Oleinik, 2022. "Relevance in Web search: between content, authority and popularity," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(1), pages 173-194, February.
    15. Aurora González-Teruel & Gregorio González-Alcaide & Maite Barrios & María-Francisca Abad-García, 2015. "Mapping recent information behavior research: an analysis of co-authorship and co-citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 687-705, May.
    16. Olof Sundin & Dirk Lewandowski & Jutta Haider, 2022. "Whose relevance? Web search engines as multisided relevance machines," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(5), pages 637-642, May.
    17. Chuhao Wu & He Zhang & John M. Carroll, 2024. "AI Governance in Higher Education: Case Studies of Guidance at Big Ten Universities," Future Internet, MDPI, vol. 16(10), pages 1-19, September.
    18. Gineke Wiggers & Suzan Verberne & Wouter van Loon & Gerrit‐Jan Zwenne, 2023. "Bibliometric‐enhanced legal information retrieval: Combining usage and citations as flavors of impact relevance," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(8), pages 1010-1025, August.
    19. P. K. Paul & A. Bhuimali & R. Rajesh & K. L. Dangwal & P. Das & J. Ganguly, 2016. "Green Computing for Eco Enriched Information Services and Systems: Environmental & Bio Informatics Perspective," Journal of Biotechnology Research, Academic Research Publishing Group, vol. 2(6), pages 44-48, 06-2016.
    20. Adan Ortiz-Cordova & Bernard J. Jansen, 2012. "Classifying web search queries to identify high revenue generating customers," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(7), pages 1426-1441, July.
    21. Juan Xie & Hongru Lu & Lele Kang & Ying Cheng, 2022. "Citing criteria and its effects on researcher's intention to cite: A mixed‐method study," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(8), pages 1079-1091, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jinfst:v:68:y:2017:i:12:p:2743-2754. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.asis.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.