Image retrieval by color semantics with incomplete knowledge

Retrieval by content from image databases faces the distance between low‐level syntactic features that can be automatically detected by conventional image processing tools and high level semantics which captures user's filtering intentions. A system is presented which bridges this gap by resorting to a theory formulated by Johannes Itten in 1960, and widely accepted in the community of fine arts, to support objective interpretation of color arrangements over paintings. The system relies upon a schema distinguishing archiving, querying, and retrieval stages. In the archiving stage, images are associated with a description capturing the spatial arrangement of regions with homogeneous chromatic attributes, as detected by the use of an automatic image processing tool. Imprecise descriptions are supported through the adoption of a hierarchical index providing a multi‐resolution representation of image contents. In the querying stage, a visual iconic language allows the expression of sentences about chromatic contents in accordance with a high‐level semantic model of colors combinations. By permitting flexible expression of abstract, non‐literal, properties, the model supports intentional vagueness and incompleteness in the specification of searching queries. In the retrieval stage, a similarity score is introduced, which accounts for the degree with which a query assertion applies to a given image. The measure of similarity drives the traversal of the hierarchical index up to find the minimum level of description precision, permitting a definite decision about the satisfaction of the query on each stored image. © 1998 John Wiley & Sons, Inc.