IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0000943.html
   My bibliography  Save this article

Causal Inference in Multisensory Perception

Author

Listed:
  • Konrad P Körding
  • Ulrik Beierholm
  • Wei Ji Ma
  • Steven Quartz
  • Joshua B Tenenbaum
  • Ladan Shams

Abstract

Perceptual events derive their significance to an animal from their meaning about the world, that is from the information they carry about their causes. The brain should thus be able to efficiently infer the causes underlying our sensory events. Here we use multisensory cue combination to study causal inference in perception. We formulate an ideal-observer model that infers whether two sensory cues originate from the same location and that also estimates their location(s). This model accurately predicts the nonlinear integration of cues by human subjects in two auditory-visual localization tasks. The results show that indeed humans can efficiently infer the causal structure as well as the location of causes. By combining insights from the study of causal inference with the ideal-observer approach to sensory cue combination, we show that the capacity to infer causal structure is not limited to conscious, high-level cognition; it is also performed continually and effortlessly in perception.

Suggested Citation

  • Konrad P Körding & Ulrik Beierholm & Wei Ji Ma & Steven Quartz & Joshua B Tenenbaum & Ladan Shams, 2007. "Causal Inference in Multisensory Perception," PLOS ONE, Public Library of Science, vol. 2(9), pages 1-10, September.
  • Handle: RePEc:plo:pone00:0000943
    DOI: 10.1371/journal.pone.0000943
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0000943
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0000943&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0000943?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Marc O. Ernst & Martin S. Banks, 2002. "Humans integrate visual and haptic information in a statistically optimal fashion," Nature, Nature, vol. 415(6870), pages 429-433, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Christoph Kayser & Ladan Shams, 2015. "Multisensory Causal Inference in the Brain," PLOS Biology, Public Library of Science, vol. 13(2), pages 1-7, February.
    2. Sophie Smit & Anina N Rich & Regine Zopf, 2019. "Visual body form and orientation cues do not modulate visuo-tactile temporal integration," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-20, December.
    3. Wendy J Adams, 2016. "The Development of Audio-Visual Integration for Temporal Judgements," PLOS Computational Biology, Public Library of Science, vol. 12(4), pages 1-17, April.
    4. Jeroen Atsma & Femke Maij & Mathieu Koppen & David E Irwin & W Pieter Medendorp, 2016. "Causal Inference for Spatial Constancy across Saccades," PLOS Computational Biology, Public Library of Science, vol. 12(3), pages 1-20, March.
    5. Tim Genewein & Eduard Hez & Zeynab Razzaghpanah & Daniel A Braun, 2015. "Structure Learning in Bayesian Sensorimotor Integration," PLOS Computational Biology, Public Library of Science, vol. 11(8), pages 1-27, August.
    6. Peter W Battaglia & Daniel Kersten & Paul R Schrater, 2011. "How Haptic Size Sensations Improve Distance Perception," PLOS Computational Biology, Public Library of Science, vol. 7(6), pages 1-13, June.
    7. Amy A Kalia & Paul R Schrater & Gordon E Legge, 2013. "Combining Path Integration and Remembered Landmarks When Navigating without Vision," PLOS ONE, Public Library of Science, vol. 8(9), pages 1-8, September.
    8. Jacqueline M Fulvio & C Shawn Green & Paul R Schrater, 2014. "Task-Specific Response Strategy Selection on the Basis of Recent Training Experience," PLOS Computational Biology, Public Library of Science, vol. 10(1), pages 1-16, January.
    9. Luigi Acerbi & Kalpana Dokka & Dora E Angelaki & Wei Ji Ma, 2018. "Bayesian comparison of explicit and implicit causal inference strategies in multisensory heading perception," PLOS Computational Biology, Public Library of Science, vol. 14(7), pages 1-38, July.
    10. Dimitrije Marković & Jan Gläscher & Peter Bossaerts & John O’Doherty & Stefan J Kiebel, 2015. "Modeling the Evolution of Beliefs Using an Attentional Focus Mechanism," PLOS Computational Biology, Public Library of Science, vol. 11(10), pages 1-34, October.
    11. David R Wozny & Ulrik R Beierholm & Ladan Shams, 2010. "Probability Matching as a Computational Strategy Used in Perception," PLOS Computational Biology, Public Library of Science, vol. 6(8), pages 1-7, August.
    12. Cory D Bonn & Maria-Eirini Netskou & Arlette Streri & Maria Dolores de Hevia, 2019. "The association of brightness with number/duration in human newborns," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-23, October.
    13. Ksander N de Winkel & Mikhail Katliar & Heinrich H Bülthoff, 2017. "Causal Inference in Multisensory Heading Estimation," PLOS ONE, Public Library of Science, vol. 12(1), pages 1-20, January.
    14. Guido Marco Cicchini & Giovanni D’Errico & David Charles Burr, 2022. "Crowding results from optimal integration of visual targets with contextual information," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    15. Adam N Sanborn & Ulrik R Beierholm, 2016. "Fast and Accurate Learning When Making Discrete Numerical Estimates," PLOS Computational Biology, Public Library of Science, vol. 12(4), pages 1-28, April.
    16. Patricia Besson & Christophe Bourdin & Lionel Bringoux, 2011. "A Comprehensive Model of Audiovisual Perception: Both Percept and Temporal Dynamics," PLOS ONE, Public Library of Science, vol. 6(8), pages 1-11, August.
    17. Max Berniker & Martin Voss & Konrad Kording, 2010. "Learning Priors for Bayesian Computations in the Nervous System," PLOS ONE, Public Library of Science, vol. 5(9), pages 1-9, September.
    18. Jannes Jegminat & Maya A Jastrzębowska & Matthew V Pachai & Michael H Herzog & Jean-Pascal Pfister, 2020. "Bayesian regression explains how human participants handle parameter uncertainty," PLOS Computational Biology, Public Library of Science, vol. 16(5), pages 1-23, May.
    19. Pedram Daee & Maryam S Mirian & Majid Nili Ahmadabadi, 2014. "Reward Maximization Justifies the Transition from Sensory Selection at Childhood to Sensory Integration at Adulthood," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-13, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Simon Weiler & Vahid Rahmati & Marcel Isstas & Johann Wutke & Andreas Walter Stark & Christian Franke & Jürgen Graf & Christian Geis & Otto W. Witte & Mark Hübener & Jürgen Bolz & Troy W. Margrie & Kn, 2024. "A primary sensory cortical interareal feedforward inhibitory circuit for tacto-visual integration," Nature Communications, Nature, vol. 15(1), pages 1-24, December.
    2. Catarina Mendonça & Pietro Mandelli & Ville Pulkki, 2016. "Modeling the Perception of Audiovisual Distance: Bayesian Causal Inference and Other Models," PLOS ONE, Public Library of Science, vol. 11(12), pages 1-18, December.
    3. Jacques Pesnot Lerousseau & Cesare V. Parise & Marc O. Ernst & Virginie Wassenhove, 2022. "Multisensory correlation computations in the human brain identified by a time-resolved encoding model," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    4. Wen-Hao Zhang & Si Wu & Krešimir Josić & Brent Doiron, 2023. "Sampling-based Bayesian inference in recurrent circuits of stochastic spiking neurons," Nature Communications, Nature, vol. 14(1), pages 1-19, December.
    5. Marine Hainguerlot & Thibault Gajdos & Jean-Christophe Vergnaud & Vincent de Gardelle, 2023. "How Overconfidence Bias Influences Suboptimality in Perceptual Decision Making," PSE-Ecole d'économie de Paris (Postprint) hal-04197403, HAL.
    6. Adam N Sanborn & Ulrik R Beierholm, 2016. "Fast and Accurate Learning When Making Discrete Numerical Estimates," PLOS Computational Biology, Public Library of Science, vol. 12(4), pages 1-28, April.
    7. Patricia Besson & Christophe Bourdin & Lionel Bringoux, 2011. "A Comprehensive Model of Audiovisual Perception: Both Percept and Temporal Dynamics," PLOS ONE, Public Library of Science, vol. 6(8), pages 1-11, August.
    8. Anthony Renard & Evan R. Harrell & Brice Bathellier, 2022. "Olfactory modulation of barrel cortex activity during active whisking and passive whisker stimulation," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    9. Seth W. Egger & Stephen G. Lisberger, 2022. "Neural structure of a sensory decoder for motor control," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    10. Wendy J Adams, 2016. "The Development of Audio-Visual Integration for Temporal Judgements," PLOS Computational Biology, Public Library of Science, vol. 12(4), pages 1-17, April.
    11. Tim Genewein & Eduard Hez & Zeynab Razzaghpanah & Daniel A Braun, 2015. "Structure Learning in Bayesian Sensorimotor Integration," PLOS Computational Biology, Public Library of Science, vol. 11(8), pages 1-27, August.
    12. Xiaochen Zhang & Lingling Jin & Jie Zhao & Jiazhen Li & Ding-Bang Luh & Tiansheng Xia, 2022. "The Influences of Different Sensory Modalities and Cognitive Loads on Walking Navigation: A Preliminary Study," Sustainability, MDPI, vol. 14(24), pages 1-14, December.
    13. Johannes Burge & Priyank Jaini, 2017. "Accuracy Maximization Analysis for Sensory-Perceptual Tasks: Computational Improvements, Filter Robustness, and Coding Advantages for Scaled Additive Noise," PLOS Computational Biology, Public Library of Science, vol. 13(2), pages 1-32, February.
    14. Yingjie Lai & Chaemoon Yoo & Xiaomin Zhou & Younghwan Pan, 2023. "Elements of Food Service Design for Low-Carbon Tourism-Based on Dine-In Tourist Behavior and Attitudes in China," Sustainability, MDPI, vol. 15(9), pages 1-21, May.
    15. Florent Meyniel & Maxime Maheu & Stanislas Dehaene, 2016. "Human Inferences about Sequences: A Minimal Transition Probability Model," PLOS Computational Biology, Public Library of Science, vol. 12(12), pages 1-26, December.
    16. Brocas, Isabelle & Carrillo, Juan D., 2012. "From perception to action: An economic model of brain processes," Games and Economic Behavior, Elsevier, vol. 75(1), pages 81-103.
    17. Jean-François Patri & Pascal Perrier & Jean-Luc Schwartz & Julien Diard, 2018. "What drives the perceptual change resulting from speech motor adaptation? Evaluation of hypotheses in a Bayesian modeling framework," PLOS Computational Biology, Public Library of Science, vol. 14(1), pages 1-38, January.
    18. Florent Meyniel, 2020. "Brain dynamics for confidence-weighted learning," PLOS Computational Biology, Public Library of Science, vol. 16(6), pages 1-27, June.
    19. Anna Lambrechts & Vincent Walsh & Virginie van Wassenhove, 2013. "Evidence Accumulation in the Magnitude System," PLOS ONE, Public Library of Science, vol. 8(12), pages 1-10, December.
    20. Carrillo, Juan & Brocas, Isabelle, 2007. "Reason, Emotion and Information Processing in the Brain," CEPR Discussion Papers 6535, C.E.P.R. Discussion Papers.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0000943. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.