Error evaluation for stemming algorithms as clustering algorithms

This paper presents mathematical evaluation measures to characterize the effect of known erroneous performance by stemming routines, and generalizes these procedures to other types of nonstatistical clustering algorithms. When clusters, or groups of intrinsically related elements, are split into smaller groups (by under‐matching the elements), there is a loss in recall in information retrieval; larger groups (caused by over‐matching) induce a loss in precision or relevance. The magnitude of error is taken to be a function of frequencies of cluster elements. When these are words in a subject‐term index generated by a stemming algorithm, retrieval capability is also affected by the strength of the algorithm, the size and content of the stemmed index, and the number of words in a query. The present Project Intrex stemming algorithm has estimated stemming‐error losses of 4% in recall and 1% in relevance on one‐word queries; the former could be reduced to almost zero by straightforward corrections of known errors in the algorithm. An expanded probabilistic model is introduced to handle a more general case in which any element need not belong unambiguously to a single cluster. Error evaluation in document classification and thesauri also is discussed in broad terms.