IDEAS home Printed from https://ideas.repec.org/a/plo/pcbi00/1003724.html
   My bibliography  Save this article

Unsupervised Feature Learning Improves Prediction of Human Brain Activity in Response to Natural Images

Author

Listed:
  • Umut Güçlü
  • Marcel A J van Gerven

Abstract

Encoding and decoding in functional magnetic resonance imaging has recently emerged as an area of research to noninvasively characterize the relationship between stimulus features and human brain activity. To overcome the challenge of formalizing what stimulus features should modulate single voxel responses, we introduce a general approach for making directly testable predictions of single voxel responses to statistically adapted representations of ecologically valid stimuli. These representations are learned from unlabeled data without supervision. Our approach is validated using a parsimonious computational model of (i) how early visual cortical representations are adapted to statistical regularities in natural images and (ii) how populations of these representations are pooled by single voxels. This computational model is used to predict single voxel responses to natural images and identify natural images from stimulus-evoked multiple voxel responses. We show that statistically adapted low-level sparse and invariant representations of natural images better span the space of early visual cortical representations and can be more effectively exploited in stimulus identification than hand-designed Gabor wavelets. Our results demonstrate the potential of our approach to better probe unknown cortical representations.Author Summary: An important but difficult problem in contemporary cognitive neuroscience is to find what stimulus features best drive responses in the human brain. The conventional approach to solve this problem is to use descriptive encoding models that predict responses to stimulus features that are known a priori. In this study, we introduce an alternative to this approach that is independent of a priori knowledge. Instead, we use a normative encoding model that predicts responses to stimulus features that are learned from unlabeled data. We show that this normative encoding model learns sparse, topographic and invariant stimulus features from tens of thousands of grayscale natural image patches without supervision, and reproduces the population behavior of simple and complex cells. We find that these stimulus features significantly better drive blood-oxygen-level dependent hemodynamic responses in early visual areas than Gabor wavelets–the fundamental building blocks of the conventional approach. Our approach will improve our understanding of how sensory information is represented beyond early visual areas since it can theoretically find what stimulus features best drive responses in other sensory areas.

Suggested Citation

  • Umut Güçlü & Marcel A J van Gerven, 2014. "Unsupervised Feature Learning Improves Prediction of Human Brain Activity in Response to Natural Images," PLOS Computational Biology, Public Library of Science, vol. 10(8), pages 1-12, August.
  • Handle: RePEc:plo:pcbi00:1003724
    DOI: 10.1371/journal.pcbi.1003724
    as

    Download full text from publisher

    File URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003724
    Download Restriction: no

    File URL: https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1003724&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pcbi.1003724?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. R. Quian Quiroga & L. Reddy & G. Kreiman & C. Koch & I. Fried, 2005. "Invariant visual representation by single neurons in the human brain," Nature, Nature, vol. 435(7045), pages 1102-1107, June.
    2. Kendrick N. Kay & Thomas Naselaris & Ryan J. Prenger & Jack L. Gallant, 2008. "Identifying natural images from human brain activity," Nature, Nature, vol. 452(7185), pages 352-355, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jörn Diedrichsen & Nikolaus Kriegeskorte, 2017. "Representational models: A common framework for understanding encoding, pattern-component, and representational-similarity analysis," PLOS Computational Biology, Public Library of Science, vol. 13(4), pages 1-33, April.
    2. Guillermo A Cecchi & Lejian Huang & Javeria Ali Hashmi & Marwan Baliki & María V Centeno & Irina Rish & A Vania Apkarian, 2012. "Predictive Dynamics of Human Pain Perception," PLOS Computational Biology, Public Library of Science, vol. 8(10), pages 1-12, October.
    3. Zvi N. Roth & Kendrick Kay & Elisha P. Merriam, 2022. "Natural scene sampling reveals reliable coarse-scale orientation tuning in human V1," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    4. Hamed Nili & Cai Wingfield & Alexander Walther & Li Su & William Marslen-Wilson & Nikolaus Kriegeskorte, 2014. "A Toolbox for Representational Similarity Analysis," PLOS Computational Biology, Public Library of Science, vol. 10(4), pages 1-11, April.
    5. Hamed Nili & Alexander Walther & Arjen Alink & Nikolaus Kriegeskorte, 2020. "Inferring exemplar discriminability in brain representations," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-28, June.
    6. Rodrigo Quian Quiroga & Marta Boscaglia & Jacques Jonas & Hernan G. Rey & Xiaoqian Yan & Louis Maillard & Sophie Colnat-Coulbois & Laurent Koessler & Bruno Rossion, 2023. "Single neuron responses underlying face recognition in the human midfusiform face-selective cortex," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    7. Jacob M. Paul & Martijn Ackooij & Tuomas C. Cate & Ben M. Harvey, 2022. "Numerosity tuning in human association cortices and local image contrast representations in early visual cortex," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    8. Martinez-Saito, Mario, 2022. "Discrete scaling and criticality in a chain of adaptive excitable integrators," Chaos, Solitons & Fractals, Elsevier, vol. 163(C).
    9. Agustin Lage-Castellanos & Giancarlo Valente & Elia Formisano & Federico De Martino, 2019. "Methods for computing the maximum performance of computational models of fMRI responses," PLOS Computational Biology, Public Library of Science, vol. 15(3), pages 1-25, March.
    10. Kay H Brodersen & Thomas M Schofield & Alexander P Leff & Cheng Soon Ong & Ekaterina I Lomakina & Joachim M Buhmann & Klaas E Stephan, 2011. "Generative Embedding for Model-Based Classification of fMRI Data," PLOS Computational Biology, Public Library of Science, vol. 7(6), pages 1-19, June.
    11. Luca D. Kolibius & Frederic Roux & George Parish & Marije Wal & Mircea Plas & Ramesh Chelvarajah & Vijay Sawlani & David T. Rollings & Johannes D. Lang & Stephanie Gollwitzer & Katrin Walther & Rüdige, 2023. "Hippocampal neurons code individual episodic memories in humans," Nature Human Behaviour, Nature, vol. 7(11), pages 1968-1979, November.
    12. Jakub Kopal & Kuldeep Kumar & Kimia Shafighi & Karin Saltoun & Claudia Modenato & Clara A. Moreau & Guillaume Huguet & Martineau Jean-Louis & Charles-Olivier Martin & Zohra Saci & Nadine Younis & Elis, 2024. "Using rare genetic mutations to revisit structural brain asymmetry," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
    13. Shinsuke Koyama & Uri Eden & Emery Brown & Robert Kass, 2010. "Bayesian decoding of neural spike trains," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 62(1), pages 37-59, February.
    14. Kiyohito Iigaya & Sanghyun Yi & Iman A. Wahle & Sandy Tanwisuth & Logan Cross & John P. O’Doherty, 2023. "Neural mechanisms underlying the hierarchical construction of perceived aesthetic value," Nature Communications, Nature, vol. 14(1), pages 1-19, December.
    15. Ming Bo Cai & Nicolas W Schuck & Jonathan W Pillow & Yael Niv, 2019. "Representational structure or task structure? Bias in neural representational similarity analysis and a Bayesian method for reducing bias," PLOS Computational Biology, Public Library of Science, vol. 15(5), pages 1-30, May.
    16. Ghislain St-Yves & Emily J. Allen & Yihan Wu & Kendrick Kay & Thomas Naselaris, 2023. "Brain-optimized deep neural network models of human visual areas learn non-hierarchical representations," Nature Communications, Nature, vol. 14(1), pages 1-16, December.
    17. Nanyi Fei & Zhiwu Lu & Yizhao Gao & Guoxing Yang & Yuqi Huo & Jingyuan Wen & Haoyu Lu & Ruihua Song & Xin Gao & Tao Xiang & Hao Sun & Ji-Rong Wen, 2022. "Towards artificial general intelligence via a multimodal foundation model," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    18. Hanzhong Liu & Bin Yu, 2017. "Comments on: High-dimensional simultaneous inference with the bootstrap," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 26(4), pages 740-750, December.
    19. Louis Kang & Taro Toyoizumi, 2024. "Distinguishing examples while building concepts in hippocampal and artificial networks," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
    20. Ahalya Prabhakar & Todd Murphey, 2022. "Mechanical intelligence for learning embodied sensor-object relationships," Nature Communications, Nature, vol. 13(1), pages 1-8, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1003724. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.