IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0023811.html
   My bibliography  Save this article

A Comprehensive Model of Audiovisual Perception: Both Percept and Temporal Dynamics

Author

Listed:
  • Patricia Besson
  • Christophe Bourdin
  • Lionel Bringoux

Abstract

The sparse information captured by the sensory systems is used by the brain to apprehend the environment, for example, to spatially locate the source of audiovisual stimuli. This is an ill-posed inverse problem whose inherent uncertainty can be solved by jointly processing the information, as well as introducing constraints during this process, on the way this multisensory information is handled. This process and its result - the percept - depend on the contextual conditions perception takes place in. To date, perception has been investigated and modeled on the basis of either one of two of its dimensions: the percept or the temporal dynamics of the process. Here, we extend our previously proposed audiovisual perception model to predict both these dimensions to capture the phenomenon as a whole. Starting from a behavioral analysis, we use a data-driven approach to elicit a Bayesian network which infers the different percepts and dynamics of the process. Context-specific independence analyses enable us to use the model's structure to directly explore how different contexts affect the way subjects handle the same available information. Hence, we establish that, while the percepts yielded by a unisensory stimulus or by the non-fusion of multisensory stimuli may be similar, they result from different processes, as shown by their differing temporal dynamics. Moreover, our model predicts the impact of bottom-up (stimulus driven) factors as well as of top-down factors (induced by instruction manipulation) on both the perception process and the percept itself.

Suggested Citation

  • Patricia Besson & Christophe Bourdin & Lionel Bringoux, 2011. "A Comprehensive Model of Audiovisual Perception: Both Percept and Temporal Dynamics," PLOS ONE, Public Library of Science, vol. 6(8), pages 1-11, August.
  • Handle: RePEc:plo:pone00:0023811
    DOI: 10.1371/journal.pone.0023811
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0023811
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0023811&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0023811?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Konrad P Körding & Ulrik Beierholm & Wei Ji Ma & Steven Quartz & Joshua B Tenenbaum & Ladan Shams, 2007. "Causal Inference in Multisensory Perception," PLOS ONE, Public Library of Science, vol. 2(9), pages 1-10, September.
    2. Marc O. Ernst & Martin S. Banks, 2002. "Humans integrate visual and haptic information in a statistically optimal fashion," Nature, Nature, vol. 415(6870), pages 429-433, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Adam N Sanborn & Ulrik R Beierholm, 2016. "Fast and Accurate Learning When Making Discrete Numerical Estimates," PLOS Computational Biology, Public Library of Science, vol. 12(4), pages 1-28, April.
    2. Wendy J Adams, 2016. "The Development of Audio-Visual Integration for Temporal Judgements," PLOS Computational Biology, Public Library of Science, vol. 12(4), pages 1-17, April.
    3. Tim Genewein & Eduard Hez & Zeynab Razzaghpanah & Daniel A Braun, 2015. "Structure Learning in Bayesian Sensorimotor Integration," PLOS Computational Biology, Public Library of Science, vol. 11(8), pages 1-27, August.
    4. Jannes Jegminat & Maya A Jastrzębowska & Matthew V Pachai & Michael H Herzog & Jean-Pascal Pfister, 2020. "Bayesian regression explains how human participants handle parameter uncertainty," PLOS Computational Biology, Public Library of Science, vol. 16(5), pages 1-23, May.
    5. Guido Marco Cicchini & Giovanni D’Errico & David Charles Burr, 2022. "Crowding results from optimal integration of visual targets with contextual information," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    6. Peter W Battaglia & Daniel Kersten & Paul R Schrater, 2011. "How Haptic Size Sensations Improve Distance Perception," PLOS Computational Biology, Public Library of Science, vol. 7(6), pages 1-13, June.
    7. Luigi Acerbi & Kalpana Dokka & Dora E Angelaki & Wei Ji Ma, 2018. "Bayesian comparison of explicit and implicit causal inference strategies in multisensory heading perception," PLOS Computational Biology, Public Library of Science, vol. 14(7), pages 1-38, July.
    8. Ksander N de Winkel & Mikhail Katliar & Heinrich H Bülthoff, 2017. "Causal Inference in Multisensory Heading Estimation," PLOS ONE, Public Library of Science, vol. 12(1), pages 1-20, January.
    9. Christoph Kayser & Ladan Shams, 2015. "Multisensory Causal Inference in the Brain," PLOS Biology, Public Library of Science, vol. 13(2), pages 1-7, February.
    10. David R Wozny & Ulrik R Beierholm & Ladan Shams, 2010. "Probability Matching as a Computational Strategy Used in Perception," PLOS Computational Biology, Public Library of Science, vol. 6(8), pages 1-7, August.
    11. Pedram Daee & Maryam S Mirian & Majid Nili Ahmadabadi, 2014. "Reward Maximization Justifies the Transition from Sensory Selection at Childhood to Sensory Integration at Adulthood," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-13, July.
    12. Simon Weiler & Vahid Rahmati & Marcel Isstas & Johann Wutke & Andreas Walter Stark & Christian Franke & Jürgen Graf & Christian Geis & Otto W. Witte & Mark Hübener & Jürgen Bolz & Troy W. Margrie & Kn, 2024. "A primary sensory cortical interareal feedforward inhibitory circuit for tacto-visual integration," Nature Communications, Nature, vol. 15(1), pages 1-24, December.
    13. Amy A Kalia & Paul R Schrater & Gordon E Legge, 2013. "Combining Path Integration and Remembered Landmarks When Navigating without Vision," PLOS ONE, Public Library of Science, vol. 8(9), pages 1-8, September.
    14. Catarina Mendonça & Pietro Mandelli & Ville Pulkki, 2016. "Modeling the Perception of Audiovisual Distance: Bayesian Causal Inference and Other Models," PLOS ONE, Public Library of Science, vol. 11(12), pages 1-18, December.
    15. Jacques Pesnot Lerousseau & Cesare V. Parise & Marc O. Ernst & Virginie Wassenhove, 2022. "Multisensory correlation computations in the human brain identified by a time-resolved encoding model," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    16. Dimitrije Marković & Jan Gläscher & Peter Bossaerts & John O’Doherty & Stefan J Kiebel, 2015. "Modeling the Evolution of Beliefs Using an Attentional Focus Mechanism," PLOS Computational Biology, Public Library of Science, vol. 11(10), pages 1-34, October.
    17. Wen-Hao Zhang & Si Wu & Krešimir Josić & Brent Doiron, 2023. "Sampling-based Bayesian inference in recurrent circuits of stochastic spiking neurons," Nature Communications, Nature, vol. 14(1), pages 1-19, December.
    18. Marine Hainguerlot & Thibault Gajdos & Jean-Christophe Vergnaud & Vincent de Gardelle, 2023. "How Overconfidence Bias Influences Suboptimality in Perceptual Decision Making," PSE-Ecole d'économie de Paris (Postprint) hal-04197403, HAL.
    19. Anthony Renard & Evan R. Harrell & Brice Bathellier, 2022. "Olfactory modulation of barrel cortex activity during active whisking and passive whisker stimulation," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    20. Seth W. Egger & Stephen G. Lisberger, 2022. "Neural structure of a sensory decoder for motor control," Nature Communications, Nature, vol. 13(1), pages 1-13, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0023811. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.