Retrieval testing with hypergeometric document models

If one could identify the source subject areas of documents and could compute the probability that any given document came from a given source, one could apply Baye's theorem to compute the probability that a query document and any other document came from the same subject area (i.e., were related). Even correct prior probabilities could be assigned under this hypothesis by examining the whole database to obtain the probabilities with which different sources occur. While we do not know how to carry out this scheme in such a way as to account for all the information contained in documents, we show here how it may be realized in a limited way. A method of modeling the sources of documents is described which accounts for the information in global term weights. The methodology is based on the hypergeometric probability distribution. Such a source model may be fit closely to a real database and may be used to convert the real database to an abstract database in which document sources are known and model retrieval is the best retrieval possible based on model document content. We have constructed such an abstract model corresponding to a database of MEDLINE records. Tests of vector retrieval methods on the abstract model indicate they are near optimal but suggest minor improvement with correct parameter choices. Preliminary results based on a test set (human judged) from the real database support these results. © 1993 John Wiley & Sons, Inc.