Prediction Errors but Not Sharpened Signals Simulate Multivoxel fMRI Patterns during Speech Perception

Successful perception depends on combining sensory input with prior knowledge. However, the underlying mechanism by which these two sources of information are combined is unknown. In speech perception, as in other domains, two functionally distinct coding schemes have been proposed for how expectations influence representation of sensory evidence. Traditional models suggest that expected features of the speech input are enhanced or sharpened via interactive activation (Sharpened Signals). Conversely, Predictive Coding suggests that expected features are suppressed so that unexpected features of the speech input (Prediction Errors) are processed further. The present work is aimed at distinguishing between these two accounts of how prior knowledge influences speech perception. By combining behavioural, univariate, and multivariate fMRI measures of how sensory detail and prior expectations influence speech perception with computational modelling, we provide evidence in favour of Prediction Error computations. Increased sensory detail and informative expectations have additive behavioural and univariate neural effects because they both improve the accuracy of word report and reduce the BOLD signal in lateral temporal lobe regions. However, sensory detail and informative expectations have interacting effects on speech representations shown by multivariate fMRI in the posterior superior temporal sulcus. When prior knowledge was absent, increased sensory detail enhanced the amount of speech information measured in superior temporal multivoxel patterns, but with informative expectations, increased sensory detail reduced the amount of measured information. Computational simulations of Sharpened Signals and Prediction Errors during speech perception could both explain these behavioural and univariate fMRI observations. However, the multivariate fMRI observations were uniquely simulated by a Prediction Error and not a Sharpened Signal model. The interaction between prior expectation and sensory detail provides evidence for a Predictive Coding account of speech perception. Our work establishes methods that can be used to distinguish representations of Prediction Error and Sharpened Signals in other perceptual domains.Neuroimaging and computational modelling explain how the human brain uses prior expectations to improve our perception of degraded speech.Author Summary: Perception inevitably depends on combining sensory input with prior expectations. This is particularly critical for identifying degraded input. However, the underlying neural mechanism by which expectations influence sensory processing is unclear. Predictive Coding theories suggest that the brain passes forward the unexpected part of the sensory input while expected properties are suppressed (i.e., Prediction Error). However, evidence to rule out the opposite mechanism in which the expected part of the sensory input is enhanced or sharpened (i.e., Sharpening) has been lacking. In this study, we investigate the neural mechanisms by which sensory clarity and prior knowledge influence the perception of degraded speech. A univariate measure of brain activity obtained from functional magnetic resonance imaging (fMRI) is in line with both neural mechanisms (Prediction Error and Sharpening). However, combining multivariate fMRI measures with computational simulations allows us to determine the underlying mechanism. Our key finding was an interaction between sensory input and prior expectations: for unexpected speech, increasing speech clarity increases the amount of information represented in sensory brain areas. In contrast, for speech that matches prior expectations, increasing speech clarity reduces the amount of this information. Our observations are uniquely simulated by a model of speech perception that includes Prediction Errors.