The Elementary Operations of Human Vision Are Not Reducible to Template-Matching

It is generally acknowledged that biological vision presents nonlinear characteristics, yet linear filtering accounts of visual processing are ubiquitous. The template-matching operation implemented by the linear-nonlinear cascade (linear filter followed by static nonlinearity) is the most widely adopted computational tool in systems neuroscience. This simple model achieves remarkable explanatory power while retaining analytical tractability, potentially extending its reach to a wide range of systems and levels in sensory processing. The extent of its applicability to human behaviour, however, remains unclear. Because sensory stimuli possess multiple attributes (e.g. position, orientation, size), the issue of applicability may be asked by considering each attribute one at a time in relation to a family of linear-nonlinear models, or by considering all attributes collectively in relation to a specified implementation of the linear-nonlinear cascade. We demonstrate that human visual processing can operate under conditions that are indistinguishable from linear-nonlinear transduction with respect to substantially different stimulus attributes of a uniquely specified target signal with associated behavioural task. However, no specific implementation of a linear-nonlinear cascade is able to account for the entire collection of results across attributes; a satisfactory account at this level requires the introduction of a small gain-control circuit, resulting in a model that no longer belongs to the linear-nonlinear family. Our results inform and constrain efforts at obtaining and interpreting comprehensive characterizations of the human sensory process by demonstrating its inescapably nonlinear nature, even under conditions that have been painstakingly fine-tuned to facilitate template-matching behaviour and to produce results that, at some level of inspection, do conform to linear filtering predictions. They also suggest that compliance with linear transduction may be the targeted outcome of carefully crafted nonlinear circuits, rather than default behaviour exhibited by basic components.Author Summary: Any attempt to model human vision must first ask: can it be approximated by a process that linearly matches the visual stimulus with an internal template? We often take this approximation for granted without properly checking its validity. Even if we assume that the approximation is valid under specific conditions, does this mean the system operates template matching across the board? We would not know exactly in what sense and to what extent the approximation may be viable. Our results address both issues. We find that template matchers are locally applicable in relation to a wide range of conditions, providing much-needed justification for several relevant computational tools. We also find, however, that there is no sense in which the system is globally a linear template: it remains inescapably nonlinear. Our findings suggest that linear transduction is not cost-free: it is not a default building block that is used for constructing expensive nonlinear processes. Rather, linear sensory representations arise from carefully constructed nonlinear processes that strike a balanced act between the necessity to retain other important computations, and the desirability of transducing and representing the visual world on a linear scale.