IDEAS home Printed from https://ideas.repec.org/a/plo/pbio00/1002073.html
   My bibliography  Save this article

Cortical Hierarchies Perform Bayesian Causal Inference in Multisensory Perception

Author

Listed:
  • Tim Rohe
  • Uta Noppeney

Abstract

To form a veridical percept of the environment, the brain needs to integrate sensory signals from a common source but segregate those from independent sources. Thus, perception inherently relies on solving the “causal inference problem.” Behaviorally, humans solve this problem optimally as predicted by Bayesian Causal Inference; yet, the underlying neural mechanisms are unexplored. Combining psychophysics, Bayesian modeling, functional magnetic resonance imaging (fMRI), and multivariate decoding in an audiovisual spatial localization task, we demonstrate that Bayesian Causal Inference is performed by a hierarchy of multisensory processes in the human brain. At the bottom of the hierarchy, in auditory and visual areas, location is represented on the basis that the two signals are generated by independent sources (= segregation). At the next stage, in posterior intraparietal sulcus, location is estimated under the assumption that the two signals are from a common source (= forced fusion). Only at the top of the hierarchy, in anterior intraparietal sulcus, the uncertainty about the causal structure of the world is taken into account and sensory signals are combined as predicted by Bayesian Causal Inference. Characterizing the computational operations of signal interactions reveals the hierarchical nature of multisensory perception in human neocortex. It unravels how the brain accomplishes Bayesian Causal Inference, a statistical computation fundamental for perception and cognition. Our results demonstrate how the brain combines information in the face of uncertainty about the underlying causal structure of the world.The human brain uses Bayesian Causal Inference to integrate and segregate information by encoding multiple estimates of spatial relationships at different levels of the auditory and visual processing hierarchies. Read the accompanying Primer.Author Summary: How can the brain integrate signals into a veridical percept of the environment without knowing whether they pertain to same or different events? For example, I can hear a bird and I can see a bird, but is it one bird singing on the branch, or is it two birds (one sitting on the branch and the other singing in the bush)? Recent studies demonstrate that human observers solve this problem optimally as predicted by Bayesian Causal Inference; yet, the neural mechanisms remain unclear. By combining psychophysics, Bayesian modelling, functional magnetic resonance imaging (fMRI), and multivariate decoding in an audiovisual localization task, we show that Bayesian Causal Inference is performed by a neural hierarchy of multisensory processes. At the bottom of the hierarchy, in auditory and visual areas, location is represented on the basis that the two signals are generated by independent sources (= segregation). At the next stage, in posterior intraparietal sulcus, location is estimated under the assumption that the two signals are from a common source (= forced fusion). Only at the top of the hierarchy, in anterior intraparietal sulcus, the uncertainty about the world’s causal structure is taken into account and sensory signals are combined as predicted by Bayesian Causal Inference.

Suggested Citation

  • Tim Rohe & Uta Noppeney, 2015. "Cortical Hierarchies Perform Bayesian Causal Inference in Multisensory Perception," PLOS Biology, Public Library of Science, vol. 13(2), pages 1-18, February.
  • Handle: RePEc:plo:pbio00:1002073
    DOI: 10.1371/journal.pbio.1002073
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002073
    Download Restriction: no

    File URL: https://journals.plos.org/plosbiology/article/file?id=10.1371/journal.pbio.1002073&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pbio.1002073?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jacqueline P. Gottlieb & Makoto Kusunoki & Michael E. Goldberg, 1998. "The representation of visual salience in monkey parietal cortex," Nature, Nature, vol. 391(6666), pages 481-484, January.
    2. Konrad P Körding & Ulrik Beierholm & Wei Ji Ma & Steven Quartz & Joshua B Tenenbaum & Ladan Shams, 2007. "Causal Inference in Multisensory Perception," PLOS ONE, Public Library of Science, vol. 2(9), pages 1-10, September.
    3. Marc O. Ernst & Martin S. Banks, 2002. "Humans integrate visual and haptic information in a statistically optimal fashion," Nature, Nature, vol. 415(6870), pages 429-433, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Máté Aller & Uta Noppeney, 2019. "To integrate or not to integrate: Temporal dynamics of hierarchical Bayesian causal inference," PLOS Biology, Public Library of Science, vol. 17(4), pages 1-31, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Adam N Sanborn & Ulrik R Beierholm, 2016. "Fast and Accurate Learning When Making Discrete Numerical Estimates," PLOS Computational Biology, Public Library of Science, vol. 12(4), pages 1-28, April.
    2. Patricia Besson & Christophe Bourdin & Lionel Bringoux, 2011. "A Comprehensive Model of Audiovisual Perception: Both Percept and Temporal Dynamics," PLOS ONE, Public Library of Science, vol. 6(8), pages 1-11, August.
    3. Wendy J Adams, 2016. "The Development of Audio-Visual Integration for Temporal Judgements," PLOS Computational Biology, Public Library of Science, vol. 12(4), pages 1-17, April.
    4. Tim Genewein & Eduard Hez & Zeynab Razzaghpanah & Daniel A Braun, 2015. "Structure Learning in Bayesian Sensorimotor Integration," PLOS Computational Biology, Public Library of Science, vol. 11(8), pages 1-27, August.
    5. Jennifer Laura Lee & Wei Ji Ma, 2021. "Point-estimating observer models for latent cause detection," PLOS Computational Biology, Public Library of Science, vol. 17(10), pages 1-29, October.
    6. Jannes Jegminat & Maya A Jastrzębowska & Matthew V Pachai & Michael H Herzog & Jean-Pascal Pfister, 2020. "Bayesian regression explains how human participants handle parameter uncertainty," PLOS Computational Biology, Public Library of Science, vol. 16(5), pages 1-23, May.
    7. Guido Marco Cicchini & Giovanni D’Errico & David Charles Burr, 2022. "Crowding results from optimal integration of visual targets with contextual information," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    8. Peter W Battaglia & Daniel Kersten & Paul R Schrater, 2011. "How Haptic Size Sensations Improve Distance Perception," PLOS Computational Biology, Public Library of Science, vol. 7(6), pages 1-13, June.
    9. Luigi Acerbi & Kalpana Dokka & Dora E Angelaki & Wei Ji Ma, 2018. "Bayesian comparison of explicit and implicit causal inference strategies in multisensory heading perception," PLOS Computational Biology, Public Library of Science, vol. 14(7), pages 1-38, July.
    10. Ksander N de Winkel & Mikhail Katliar & Heinrich H Bülthoff, 2017. "Causal Inference in Multisensory Heading Estimation," PLOS ONE, Public Library of Science, vol. 12(1), pages 1-20, January.
    11. Christoph Kayser & Ladan Shams, 2015. "Multisensory Causal Inference in the Brain," PLOS Biology, Public Library of Science, vol. 13(2), pages 1-7, February.
    12. David R Wozny & Ulrik R Beierholm & Ladan Shams, 2010. "Probability Matching as a Computational Strategy Used in Perception," PLOS Computational Biology, Public Library of Science, vol. 6(8), pages 1-7, August.
    13. Pedram Daee & Maryam S Mirian & Majid Nili Ahmadabadi, 2014. "Reward Maximization Justifies the Transition from Sensory Selection at Childhood to Sensory Integration at Adulthood," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-13, July.
    14. Simon Weiler & Vahid Rahmati & Marcel Isstas & Johann Wutke & Andreas Walter Stark & Christian Franke & Jürgen Graf & Christian Geis & Otto W. Witte & Mark Hübener & Jürgen Bolz & Troy W. Margrie & Kn, 2024. "A primary sensory cortical interareal feedforward inhibitory circuit for tacto-visual integration," Nature Communications, Nature, vol. 15(1), pages 1-24, December.
    15. Li Zhaoping & Li Zhe, 2015. "Primary Visual Cortex as a Saliency Map: A Parameter-Free Prediction and Its Test by Behavioral Data," PLOS Computational Biology, Public Library of Science, vol. 11(10), pages 1-39, October.
    16. Amy A Kalia & Paul R Schrater & Gordon E Legge, 2013. "Combining Path Integration and Remembered Landmarks When Navigating without Vision," PLOS ONE, Public Library of Science, vol. 8(9), pages 1-8, September.
    17. Catarina Mendonça & Pietro Mandelli & Ville Pulkki, 2016. "Modeling the Perception of Audiovisual Distance: Bayesian Causal Inference and Other Models," PLOS ONE, Public Library of Science, vol. 11(12), pages 1-18, December.
    18. Janina Klautke & Celia Foster & W. Pieter Medendorp & Tobias Heed, 2023. "Dynamic spatial coding in parietal cortex mediates tactile-motor transformation," Nature Communications, Nature, vol. 14(1), pages 1-18, December.
    19. Jacques Pesnot Lerousseau & Cesare V. Parise & Marc O. Ernst & Virginie Wassenhove, 2022. "Multisensory correlation computations in the human brain identified by a time-resolved encoding model," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    20. Dimitrije Marković & Jan Gläscher & Peter Bossaerts & John O’Doherty & Stefan J Kiebel, 2015. "Modeling the Evolution of Beliefs Using an Attentional Focus Mechanism," PLOS Computational Biology, Public Library of Science, vol. 11(10), pages 1-34, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pbio00:1002073. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosbiology (email available below). General contact details of provider: https://journals.plos.org/plosbiology/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.