IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-45679-0.html
   My bibliography  Save this article

Improved modeling of human vision by incorporating robustness to blur in convolutional neural networks

Author

Listed:
  • Hojin Jang

    (Vanderbilt Vision Research Center, Vanderbilt University
    Massachusetts Institute of Technology
    Korea University)

  • Frank Tong

    (Vanderbilt Vision Research Center, Vanderbilt University)

Abstract

Whenever a visual scene is cast onto the retina, much of it will appear degraded due to poor resolution in the periphery; moreover, optical defocus can cause blur in central vision. However, the pervasiveness of blurry or degraded input is typically overlooked in the training of convolutional neural networks (CNNs). We hypothesized that the absence of blurry training inputs may cause CNNs to rely excessively on high spatial frequency information for object recognition, thereby causing systematic deviations from biological vision. We evaluated this hypothesis by comparing standard CNNs with CNNs trained on a combination of clear and blurry images. We show that blur-trained CNNs outperform standard CNNs at predicting neural responses to objects across a variety of viewing conditions. Moreover, blur-trained CNNs acquire increased sensitivity to shape information and greater robustness to multiple forms of visual noise, leading to improved correspondence with human perception. Our results provide multi-faceted neurocomputational evidence that blurry visual experiences may be critical for conferring robustness to biological visual systems.

Suggested Citation

  • Hojin Jang & Frank Tong, 2024. "Improved modeling of human vision by incorporating robustness to blur in convolutional neural networks," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-45679-0
    DOI: 10.1038/s41467-024-45679-0
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-45679-0
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-45679-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Timothy D. Oleskiw & Amy Nowack & Anitha Pasupathy, 2018. "Joint coding of shape and blur in area V4," Nature Communications, Nature, vol. 9(1), pages 1-13, December.
    2. Tomoyasu Horikawa & Yukiyasu Kamitani, 2017. "Generic decoding of seen and imagined objects using hierarchical visual features," Nature Communications, Nature, vol. 8(1), pages 1-15, August.
    3. J. Gold & P. J. Bennett & A. B. Sekuler, 1999. "Signal but not noise changes with perceptual learning," Nature, Nature, vol. 402(6758), pages 176-178, November.
    4. Pinglei Bao & Liang She & Mason McGill & Doris Y. Tsao, 2020. "A map of object space in primate inferotemporal cortex," Nature, Nature, vol. 583(7814), pages 103-108, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vasiliki Bougou & Michaël Vanhoyland & Alexander Bertrand & Wim Paesschen & Hans Op De Beeck & Peter Janssen & Tom Theys, 2024. "Neuronal tuning and population representations of shape and category in human visual cortex," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    2. Saloni Sharma & Kasper Vinken & Akshay V. Jagadeesh & Margaret S. Livingstone, 2024. "Face cells encode object parts more than facial configuration of illusory faces," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    3. Ari S. Benjamin & Ling-Qi Zhang & Cheng Qiu & Alan A. Stocker & Konrad P. Kording, 2022. "Efficient neural codes naturally emerge through gradient descent learning," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    4. Guohua Shen & Tomoyasu Horikawa & Kei Majima & Yukiyasu Kamitani, 2019. "Deep image reconstruction from human brain activity," PLOS Computational Biology, Public Library of Science, vol. 15(1), pages 1-23, January.
    5. Richard F Murray & Khushbu Patel & Alan Yee, 2015. "Posterior Probability Matching and Human Perceptual Decision Making," PLOS Computational Biology, Public Library of Science, vol. 11(6), pages 1-16, June.
    6. Mengna Yao & Bincheng Wen & Mingpo Yang & Jiebin Guo & Haozhou Jiang & Chao Feng & Yilei Cao & Huiguang He & Le Chang, 2023. "High-dimensional topographic organization of visual features in the primate temporal lobe," Nature Communications, Nature, vol. 14(1), pages 1-23, December.
    7. Elia Shahbazi & Timothy Ma & Martin Pernuš & Walter Scheirer & Arash Afraz, 2024. "Perceptography unveils the causal contribution of inferior temporal cortex to visual perception," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    8. Laurent Caplette & Nicholas B. Turk-Browne, 2024. "Computational reconstruction of mental representations using human behavior," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
    9. Alan Consorti & Gabriele Sansevero & Irene Marco & Silvia Floridia & Elena Novelli & Nicoletta Berardi & Alessandro Sale, 2024. "An essential role for the latero-medial secondary visual cortex in the acquisition and retention of visual perceptual learning in mice," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    10. Olivia Rose & James Johnson & Binxu Wang & Carlos R. Ponce, 2021. "Visual prototypes in the ventral stream are attuned to complexity and gaze behavior," Nature Communications, Nature, vol. 12(1), pages 1-16, December.
    11. Serra E. Favila & Brice A. Kuhl & Jonathan Winawer, 2022. "Perception and memory have distinct spatial tuning properties in human visual cortex," Nature Communications, Nature, vol. 13(1), pages 1-21, December.
    12. Ian J C MacCormick & Bryan M Williams & Yalin Zheng & Kun Li & Baidaa Al-Bander & Silvester Czanner & Rob Cheeseman & Colin E Willoughby & Emery N Brown & George L Spaeth & Gabriela Czanner, 2019. "Accurate, fast, data efficient and interpretable glaucoma diagnosis with automated spatial analysis of the whole cup to disc profile," PLOS ONE, Public Library of Science, vol. 14(1), pages 1-20, January.
    13. Benjamin Lahner & Kshitij Dwivedi & Polina Iamshchinina & Monika Graumann & Alex Lascelles & Gemma Roig & Alessandro Thomas Gifford & Bowen Pan & SouYoung Jin & N. Apurva Ratan Murty & Kendrick Kay & , 2024. "Modeling short visual events through the BOLD moments video fMRI dataset and metadata," Nature Communications, Nature, vol. 15(1), pages 1-26, December.
    14. Daniel Pacheco-Estefan & Marie-Christin Fellner & Lukas Kunz & Hui Zhang & Peter Reinacher & Charlotte Roy & Armin Brandt & Andreas Schulze-Bonhage & Linglin Yang & Shuang Wang & Jing Liu & Gui Xue & , 2024. "Maintenance and transformation of representational formats during working memory prioritization," Nature Communications, Nature, vol. 15(1), pages 1-20, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-45679-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.