Tag‐based social image retrieval: An empirical evaluation

Tags associated with social images are valuable information source for superior image search and retrieval experiences. Although various heuristics are valuable to boost tag‐based search for images, there is a lack of general framework to study the impact of these heuristics. Specifically, the task of ranking images matching a given tag query based on their associated tags in descending order of relevance has not been well studied. In this article, we take the first step to propose a generic, flexible, and extensible framework for this task and exploit it for a systematic and comprehensive empirical evaluation of various methods for ranking images. To this end, we identified five orthogonal dimensions to quantify the matching score between a tagged image and a tag query. These five dimensions are: (i) tag relatedness to measure the degree of effectiveness of a tag describing the tagged image; (ii) tag discrimination to quantify the degree of discrimination of a tag with respect to the entire tagged image collection; (iii) tag length normalization analogous to document length normalization in web search; (iv) tag‐query matching model for the matching score computation between an image tag and a query tag; and (v) query model for tag query rewriting. For each dimension, we identify a few implementations and evaluate their impact on NUS‐WIDE dataset, the largest human‐annotated dataset consisting of more than 269K tagged images from Flickr. We evaluated 81 single‐tag queries and 443 multi‐tag queries over 288 search methods and systematically compare their performances using standard metrics including Precision at top‐K, Mean Average Precision (MAP), Recall, and Normalized Discounted Cumulative Gain (NDCG).