Adaptive comparison matrix: An efficient method for psychological scaling of large stimulus sets

Studies on natural and social vision often need to quantify subjective intensity along a particular dimension for a large number of stimuli whose perceptual ordering is unknown. Here, we introduce an easy experimental protocol of comparative judgments that can rank and scale subjective stimulus intensity using a comparatively small number of trials. On each trial in our protocol, the observer initially views M stimuli sampled from a space of N stimuli and selects the stimulus that elicits maximum subjective response along a given dimension (e.g., the most attractive). The selected stimulus is subsequently discarded, the observer then performs a judgment on the remaining stimuli, and the process is iterated until the last stimulus remains and a new trial begins. The method relies on sorting perceived stimulus order in the N x N comparison matrix via logistic regression and sampling the next set of M stimuli such that responses will be collected only for stimulus pairs whose expected response ratio is most informative. Numerical simulations demonstrate that this method can estimate psychological scale with a small number of responses. Psychophysical experiments confirm that the method can quickly estimate the contrast response function for gratings and the perceived glossiness of naturalistic objects. This protocol would be useful for characterizing human judgments along various dimensions, especially those with no physical image correlates such as emotional and social attributes.