The relative contribution of color and material in object selection

Object perception is inherently multidimensional: information about color, material, texture and shape all guide how we interact with objects. We developed a paradigm that quantifies how two object properties (color and material) combine in object selection. On each experimental trial, observers viewed three blob-shaped objects—the target and two tests—and selected the test that was more similar to the target. Across trials, the target object was fixed, while the tests varied in color (across 7 levels) and material (also 7 levels, yielding 49 possible stimuli). We used an adaptive trial selection procedure (Quest+) to present, on each trial, the stimulus test pair that is most informative of underlying processes that drive selection. We present a novel computational model that allows us to describe observers’ selection data in terms of (1) the underlying perceptual stimulus representation and (2) a color-material weight, which quantifies the relative importance of color vs. material in selection. We document large individual differences in the color-material weight across the 12 observers we tested. Furthermore, our analyses reveal limits on how precisely selection data simultaneously constrain perceptual representations and the color-material weight. These limits should guide future efforts towards understanding the multidimensional nature of object perception.Author summary: Much is known about how the visual system extracts information about individual object properties, such as color or material. Considerably less is known about how percepts of these properties interact to form a multidimensional object representation. We report the first quantitative analysis of how perceived color and material combine in object selection, using a task designed to reflect key aspects of how we use vision in real life. We introduce a computational model that describes observers’ selection behavior in terms of (1) how objects are represented in an underlying subjective perceptual color-material space and (2) how differences in perceived object color and material combine to guide selection. We find large individual differences in the degree to which observers select objects based on color relative to material: some base their selections almost entirely on color, some weight color and material nearly equally, and others rely almost entirely on material. A fine-grained analysis clarifies the limits on how precisely selection data may be leveraged to simultaneously understand the underlying perceptual representations on one hand and how the information about perceived color and material combine on the other. Our work provides a foundation for improving our understanding of visual computations in natural viewing.