Material and shape perception based on two types of intensity gradient information

Visual estimation of the material and shape of an object from a single image includes a hard ill-posed computational problem. However, in our daily life we feel we can estimate both reasonably well. The neural computation underlying this ability remains poorly understood. Here we propose that the human visual system uses different aspects of object images to separately estimate the contributions of the material and shape. Specifically, material perception relies mainly on the intensity gradient magnitude information, while shape perception relies mainly on the intensity gradient order information. A clue to this hypothesis was provided by the observation that luminance-histogram manipulation, which changes luminance gradient magnitudes but not the luminance-order map, effectively alters the material appearance but not the shape of an object. In agreement with this observation, we found that the simulated physical material changes do not significantly affect the intensity order information. A series of psychophysical experiments further indicate that human surface shape perception is robust against intensity manipulations provided they do not disturb the intensity order information. In addition, we show that the two types of gradient information can be utilized for the discrimination of albedo changes from highlights. These findings suggest that the visual system relies on these diagnostic image features to estimate physical properties in a distal world.Author summary: Objects in our visual world contain a variety of material information. Although such information enables us to experience rich material impressions, it can be a distraction for the estimation of other physical properties such as shapes, albedos, and illuminations. The coupled estimation of these properties we humans perform in daily situations is one of the fundamental mysteries in visual neuroscience. Here, we show that material and shape perception depend on two different types of intensity gradient information. Specifically, our image analyses and psychophysical experiments show that a human’s material perception relies mainly on the intensity gradient magnitude information, while shape perception relies mainly on the intensity gradient order information. In addition, we show that the intensity order information can be utilized for discriminating albedo changes on an object surface from other physical properties including specular highlights.