To integrate or not to integrate: Temporal dynamics of hierarchical Bayesian causal inference

To form a percept of the environment, the brain needs to solve the binding problem—inferring whether signals come from a common cause and are integrated or come from independent causes and are segregated. Behaviourally, humans solve this problem near-optimally as predicted by Bayesian causal inference; but the neural mechanisms remain unclear. Combining Bayesian modelling, electroencephalography (EEG), and multivariate decoding in an audiovisual spatial localisation task, we show that the brain accomplishes Bayesian causal inference by dynamically encoding multiple spatial estimates. Initially, auditory and visual signal locations are estimated independently; next, an estimate is formed that combines information from vision and audition. Yet, it is only from 200 ms onwards that the brain integrates audiovisual signals weighted by their bottom-up sensory reliabilities and top-down task relevance into spatial priority maps that guide behavioural responses. As predicted by Bayesian causal inference, these spatial priority maps take into account the brain’s uncertainty about the world’s causal structure and flexibly arbitrate between sensory integration and segregation. The dynamic evolution of perceptual estimates thus reflects the hierarchical nature of Bayesian causal inference, a statistical computation, which is crucial for effective interactions with the environment.This electroencephalography-based study temporally resolves how the human brain uses Bayesian Causal Inference in perception, unravelling how the brain arbitrates between information integration and segregation by dynamically encoding multiple perceptual estimates.Author summary: The ability to tell whether various sensory signals come from the same or different sources is essential for forming a coherent percept of the environment. For example, when crossing a busy road at dusk, seeing and hearing an approaching car helps us estimate its location better, but only if its visual image is associated—correctly—with its sound and not with the sound of a different car far away. This is the so-called binding problem, and numerous studies have demonstrated that humans solve this near-optimally as predicted by Bayesian causal inference; however, the underlying neural mechanisms remain unclear. We combined Bayesian modelling, electroencephalography (EEG), and multivariate decoding in an audiovisual spatial localisation task to show that the brain dynamically encodes multiple spatial estimates while accomplishing Bayesian causal inference. First, auditory and visual signal locations are estimated independently; next, information from vision and audition is combined. Finally, from 200 ms onwards, the brain weights audiovisual signals by their sensory reliabilities and task relevance to guide behavioural responses as predicted by Bayesian causal inference.