IDEAS home Printed from https://ideas.repec.org/a/nat/nathum/v5y2021i10d10.1038_s41562-021-01097-6.html
   My bibliography  Save this article

Unsupervised learning predicts human perception and misperception of gloss

Author

Listed:
  • Katherine R. Storrs

    (Justus Liebig University Giessen)

  • Barton L. Anderson

    (University of Sydney)

  • Roland W. Fleming

    (Justus Liebig University Giessen
    University of Marburg and Justus Liebig University Giessen)

Abstract

Reflectance, lighting and geometry combine in complex ways to create images. How do we disentangle these to perceive individual properties, such as surface glossiness? We suggest that brains disentangle properties by learning to model statistical structure in proximal images. To test this hypothesis, we trained unsupervised generative neural networks on renderings of glossy surfaces and compared their representations with human gloss judgements. The networks spontaneously cluster images according to distal properties such as reflectance and illumination, despite receiving no explicit information about these properties. Intriguingly, the resulting representations also predict the specific patterns of ‘successes’ and ‘errors’ in human perception. Linearly decoding specular reflectance from the model’s internal code predicts human gloss perception better than ground truth, supervised networks or control models, and it predicts, on an image-by-image basis, illusions of gloss perception caused by interactions between material, shape and lighting. Unsupervised learning may underlie many perceptual dimensions in vision and beyond.

Suggested Citation

  • Katherine R. Storrs & Barton L. Anderson & Roland W. Fleming, 2021. "Unsupervised learning predicts human perception and misperception of gloss," Nature Human Behaviour, Nature, vol. 5(10), pages 1402-1417, October.
  • Handle: RePEc:nat:nathum:v:5:y:2021:i:10:d:10.1038_s41562-021-01097-6
    DOI: 10.1038/s41562-021-01097-6
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41562-021-01097-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1038/s41562-021-01097-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Stelios M. Smirnakis & Michael J. Berry & David K. Warland & William Bialek & Markus Meister, 1997. "Adaptation of retinal processing to image contrast and spatial scale," Nature, Nature, vol. 386(6620), pages 69-73, March.
    2. Alberto Testolin & Ivilin Stoianov & Marco Zorzi, 2017. "Publisher Correction: Letter perception emerges from unsupervised deep learning and recycling of natural image features," Nature Human Behaviour, Nature, vol. 1(11), pages 843-843, November.
    3. Hamed Nili & Cai Wingfield & Alexander Walther & Li Su & William Marslen-Wilson & Nikolaus Kriegeskorte, 2014. "A Toolbox for Representational Similarity Analysis," PLOS Computational Biology, Public Library of Science, vol. 10(4), pages 1-11, April.
    4. Johannes Mehrer & Courtney J. Spoerer & Nikolaus Kriegeskorte & Tim C. Kietzmann, 2020. "Individual differences among deep neural network models," Nature Communications, Nature, vol. 11(1), pages 1-12, December.
    5. Isamu Motoyoshi & Shin'ya Nishida & Lavanya Sharan & Edward H. Adelson, 2007. "Image statistics and the perception of surface qualities," Nature, Nature, vol. 447(7141), pages 206-209, May.
    6. Masataka Sawayama & Shin'ya Nishida, 2018. "Material and shape perception based on two types of intensity gradient information," PLOS Computational Biology, Public Library of Science, vol. 14(4), pages 1-40, April.
    7. Alberto Testolin & Ivilin Stoianov & Marco Zorzi, 2017. "Letter perception emerges from unsupervised deep learning and recycling of natural image features," Nature Human Behaviour, Nature, vol. 1(9), pages 657-664, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alexandra C. Schmid & Pascal Barla & Katja Doerschner, 2023. "Material category of visual objects computed from specular image structure," Nature Human Behaviour, Nature, vol. 7(7), pages 1152-1169, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Valentina Krenz & Arjen Alink & Tobias Sommer & Benno Roozendaal & Lars Schwabe, 2023. "Time-dependent memory transformation in hippocampus and neocortex is semantic in nature," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    2. Mark L Ioffe & Michael J Berry II, 2017. "The structured ‘low temperature’ phase of the retinal population code," PLOS Computational Biology, Public Library of Science, vol. 13(10), pages 1-31, October.
    3. Hamed Nili & Alexander Walther & Arjen Alink & Nikolaus Kriegeskorte, 2020. "Inferring exemplar discriminability in brain representations," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-28, June.
    4. Masataka Sawayama & Shin'ya Nishida, 2018. "Material and shape perception based on two types of intensity gradient information," PLOS Computational Biology, Public Library of Science, vol. 14(4), pages 1-40, April.
    5. Jason S Prentice & Olivier Marre & Mark L Ioffe & Adrianna R Loback & Gašper Tkačik & Michael J Berry II, 2016. "Error-Robust Modes of the Retinal Population Code," PLOS Computational Biology, Public Library of Science, vol. 12(11), pages 1-32, November.
    6. Tristan G. Heintz & Antonio J. Hinojosa & Sina E. Dominiak & Leon Lagnado, 2022. "Opposite forms of adaptation in mouse visual cortex are controlled by distinct inhibitory microcircuits," Nature Communications, Nature, vol. 13(1), pages 1-14, December.
    7. Sreejan Kumar & Theodore R. Sumers & Takateru Yamakoshi & Ariel Goldstein & Uri Hasson & Kenneth A. Norman & Thomas L. Griffiths & Robert D. Hawkins & Samuel A. Nastase, 2024. "Shared functional specialization in transformer-based language models and the human brain," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
    8. Tony Vladusich & Marcel P Lucassen & Frans W Cornelissen, 2007. "Brightness and Darkness as Perceptual Dimensions," PLOS Computational Biology, Public Library of Science, vol. 3(10), pages 1-10, October.
    9. Ming Bo Cai & Nicolas W Schuck & Jonathan W Pillow & Yael Niv, 2019. "Representational structure or task structure? Bias in neural representational similarity analysis and a Bayesian method for reducing bias," PLOS Computational Biology, Public Library of Science, vol. 15(5), pages 1-30, May.
    10. Cai Wingfield & Li Su & Xunying Liu & Chao Zhang & Phil Woodland & Andrew Thwaites & Elisabeth Fonteneau & William D Marslen-Wilson, 2017. "Relating dynamic brain states to dynamic machine states: Human and machine solutions to the speech recognition problem," PLOS Computational Biology, Public Library of Science, vol. 13(9), pages 1-25, September.
    11. Michael F Bonner & Russell A Epstein, 2018. "Computational mechanisms underlying cortical responses to the affordance properties of visual scenes," PLOS Computational Biology, Public Library of Science, vol. 14(4), pages 1-31, April.
    12. Miguel Maravall & Rasmus S Petersen & Adrienne L Fairhall & Ehsan Arabzadeh & Mathew E Diamond, 2007. "Shifts in Coding Properties and Maintenance of Information Transmission during Adaptation in Barrel Cortex," PLOS Biology, Public Library of Science, vol. 5(2), pages 1-12, January.
    13. Sebastian P. H. Speer & Laetitia Mwilambwe-Tshilobo & Lily Tsoi & Shannon M. Burns & Emily B. Falk & Diana I. Tamir, 2024. "Hyperscanning shows friends explore and strangers converge in conversation," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    14. Máté Aller & Agoston Mihalik & Uta Noppeney, 2022. "Audiovisual adaptation is expressed in spatial and decisional codes," Nature Communications, Nature, vol. 13(1), pages 1-17, December.
    15. Jörn Diedrichsen & Nikolaus Kriegeskorte, 2017. "Representational models: A common framework for understanding encoding, pattern-component, and representational-similarity analysis," PLOS Computational Biology, Public Library of Science, vol. 13(4), pages 1-33, April.
    16. Marcus H C Howlett & Robert G Smith & Maarten Kamermans, 2017. "A novel mechanism of cone photoreceptor adaptation," PLOS Biology, Public Library of Science, vol. 15(4), pages 1-28, April.
    17. Jian K Liu & Tim Gollisch, 2015. "Spike-Triggered Covariance Analysis Reveals Phenomenological Diversity of Contrast Adaptation in the Retina," PLOS Computational Biology, Public Library of Science, vol. 11(7), pages 1-30, July.
    18. Christianne Jacobs & Kirsten Petras & Pieter Moors & Valerie Goffaux, 2020. "Contrast versus identity encoding in the face image follow distinct orientation selectivity profiles," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-22, March.
    19. Satoko Amemori & Ann M. Graybiel & Ken-ichi Amemori, 2024. "Cingulate microstimulation induces negative decision-making via reduced top-down influence on primate fronto-cingulo-striatal network," Nature Communications, Nature, vol. 15(1), pages 1-17, December.
    20. Julia Berezutskaya & Zachary V Freudenburg & Umut Güçlü & Marcel A J van Gerven & Nick F Ramsey, 2020. "Brain-optimized extraction of complex sound features that drive continuous auditory perception," PLOS Computational Biology, Public Library of Science, vol. 16(7), pages 1-34, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nathum:v:5:y:2021:i:10:d:10.1038_s41562-021-01097-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.