IDEAS home Printed from https://ideas.repec.org/a/nat/nathum/v8y2024i11d10.1038_s41562-024-01980-y.html
   My bibliography  Save this article

Distributed representations of behaviour-derived object dimensions in the human visual system

Author

Listed:
  • Oliver Contier

    (Max Planck Institute for Human Cognitive and Brain Sciences
    Max Planck School of Cognition)

  • Chris I. Baker

    (National Institutes of Health)

  • Martin N. Hebart

    (Max Planck Institute for Human Cognitive and Brain Sciences
    Justus Liebig University Giessen)

Abstract

Object vision is commonly thought to involve a hierarchy of brain regions processing increasingly complex image features, with high-level visual cortex supporting object recognition and categorization. However, object vision supports diverse behavioural goals, suggesting basic limitations of this category-centric framework. To address these limitations, we mapped a series of dimensions derived from a large-scale analysis of human similarity judgements directly onto the brain. Our results reveal broadly distributed representations of behaviourally relevant information, demonstrating selectivity to a wide variety of novel dimensions while capturing known selectivities for visual features and categories. Behaviour-derived dimensions were superior to categories at predicting brain responses, yielding mixed selectivity in much of visual cortex and sparse selectivity in category-selective clusters. This framework reconciles seemingly disparate findings regarding regional specialization, explaining category selectivity as a special case of sparse response profiles among representational dimensions, suggesting a more expansive view on visual processing in the human brain.

Suggested Citation

  • Oliver Contier & Chris I. Baker & Martin N. Hebart, 2024. "Distributed representations of behaviour-derived object dimensions in the human visual system," Nature Human Behaviour, Nature, vol. 8(11), pages 2179-2193, November.
  • Handle: RePEc:nat:nathum:v:8:y:2024:i:11:d:10.1038_s41562-024-01980-y
    DOI: 10.1038/s41562-024-01980-y
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41562-024-01980-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1038/s41562-024-01980-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jeanette A Mumford & Jean-Baptiste Poline & Russell A Poldrack, 2015. "Orthogonalization of Regressors in fMRI Models," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-11, April.
    2. Martin N. Hebart & Charles Y. Zheng & Francisco Pereira & Chris I. Baker, 2020. "Revealing the multidimensional mental representations of natural objects underlying human similarity judgements," Nature Human Behaviour, Nature, vol. 4(11), pages 1173-1185, November.
    3. Kendrick N. Kay & Thomas Naselaris & Ryan J. Prenger & Jack L. Gallant, 2008. "Identifying natural images from human brain activity," Nature, Nature, vol. 452(7185), pages 352-355, March.
    4. Pinglei Bao & Liang She & Mason McGill & Doris Y. Tsao, 2020. "A map of object space in primate inferotemporal cortex," Nature, Nature, vol. 583(7814), pages 103-108, July.
    5. Russell Epstein & Nancy Kanwisher, 1998. "A cortical representation of the local visual environment," Nature, Nature, vol. 392(6676), pages 598-601, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vasiliki Bougou & Michaël Vanhoyland & Alexander Bertrand & Wim Paesschen & Hans Op De Beeck & Peter Janssen & Tom Theys, 2024. "Neuronal tuning and population representations of shape and category in human visual cortex," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    2. Mengna Yao & Bincheng Wen & Mingpo Yang & Jiebin Guo & Haozhou Jiang & Chao Feng & Yilei Cao & Huiguang He & Le Chang, 2023. "High-dimensional topographic organization of visual features in the primate temporal lobe," Nature Communications, Nature, vol. 14(1), pages 1-23, December.
    3. Laurent Caplette & Nicholas B. Turk-Browne, 2024. "Computational reconstruction of mental representations using human behavior," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
    4. Samy A. Abdel-Ghaffar & Alexander G. Huth & Mark D. Lescroart & Dustin Stansbury & Jack L. Gallant & Sonia J. Bishop, 2024. "Occipital-temporal cortical tuning to semantic and affective features of natural images predicts associated behavioral responses," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    5. Johannes Haushofer & Margaret S Livingstone & Nancy Kanwisher, 2008. "Multivariate Patterns in Object-Selective Cortex Dissociate Perceptual and Physical Shape Similarity," PLOS Biology, Public Library of Science, vol. 6(7), pages 1-9, July.
    6. Marisa Nordt & Jesse Gomez & Vaidehi S. Natu & Alex A. Rezai & Dawn Finzi & Holly Kular & Kalanit Grill-Spector, 2023. "Longitudinal development of category representations in ventral temporal cortex predicts word and face recognition," Nature Communications, Nature, vol. 14(1), pages 1-16, December.
    7. Ying Wang & Xue Zhang & Chunhui Wang & Weifen Huang & Qian Xu & Dong Liu & Wen Zhou & Shanguang Chen & Yi Jiang, 2022. "Modulation of biological motion perception in humans by gravity," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    8. Joel Z Leibo & Qianli Liao & Fabio Anselmi & Tomaso Poggio, 2015. "The Invariance Hypothesis Implies Domain-Specific Regions in Visual Cortex," PLOS Computational Biology, Public Library of Science, vol. 11(10), pages 1-29, October.
    9. Isabella C. Wagner & Luise P. Graichen & Boryana Todorova & Andre Lüttig & David B. Omer & Matthias Stangl & Claus Lamm, 2023. "Entorhinal grid-like codes and time-locked network dynamics track others navigating through space," Nature Communications, Nature, vol. 14(1), pages 1-18, December.
    10. Umut Güçlü & Marcel A J van Gerven, 2014. "Unsupervised Feature Learning Improves Prediction of Human Brain Activity in Response to Natural Images," PLOS Computational Biology, Public Library of Science, vol. 10(8), pages 1-12, August.
    11. Saloni Sharma & Kasper Vinken & Akshay V. Jagadeesh & Margaret S. Livingstone, 2024. "Face cells encode object parts more than facial configuration of illusory faces," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    12. Guillermo A Cecchi & Lejian Huang & Javeria Ali Hashmi & Marwan Baliki & María V Centeno & Irina Rish & A Vania Apkarian, 2012. "Predictive Dynamics of Human Pain Perception," PLOS Computational Biology, Public Library of Science, vol. 8(10), pages 1-12, October.
    13. Zvi N. Roth & Kendrick Kay & Elisha P. Merriam, 2022. "Natural scene sampling reveals reliable coarse-scale orientation tuning in human V1," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    14. Hamed Nili & Cai Wingfield & Alexander Walther & Li Su & William Marslen-Wilson & Nikolaus Kriegeskorte, 2014. "A Toolbox for Representational Similarity Analysis," PLOS Computational Biology, Public Library of Science, vol. 10(4), pages 1-11, April.
    15. Hamed Nili & Alexander Walther & Arjen Alink & Nikolaus Kriegeskorte, 2020. "Inferring exemplar discriminability in brain representations," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-28, June.
    16. Jacob M. Paul & Martijn Ackooij & Tuomas C. Cate & Ben M. Harvey, 2022. "Numerosity tuning in human association cortices and local image contrast representations in early visual cortex," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    17. Laurens Winkelmeier & Carla Filosa & Renée Hartig & Max Scheller & Markus Sack & Jonathan R. Reinwald & Robert Becker & David Wolf & Martin Fungisai Gerchen & Alexander Sartorius & Andreas Meyer-Linde, 2022. "Striatal hub of dynamic and stabilized prediction coding in forebrain networks for olfactory reinforcement learning," Nature Communications, Nature, vol. 13(1), pages 1-21, December.
    18. Agustin Lage-Castellanos & Giancarlo Valente & Elia Formisano & Federico De Martino, 2019. "Methods for computing the maximum performance of computational models of fMRI responses," PLOS Computational Biology, Public Library of Science, vol. 15(3), pages 1-25, March.
    19. Kay H Brodersen & Thomas M Schofield & Alexander P Leff & Cheng Soon Ong & Ekaterina I Lomakina & Joachim M Buhmann & Klaas E Stephan, 2011. "Generative Embedding for Model-Based Classification of fMRI Data," PLOS Computational Biology, Public Library of Science, vol. 7(6), pages 1-19, June.
    20. Jaron T Colas & Wolfgang M Pauli & Tobias Larsen & J Michael Tyszka & John P O’Doherty, 2017. "Distinct prediction errors in mesostriatal circuits of the human brain mediate learning about the values of both states and actions: evidence from high-resolution fMRI," PLOS Computational Biology, Public Library of Science, vol. 13(10), pages 1-32, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nathum:v:8:y:2024:i:11:d:10.1038_s41562-024-01980-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.