Author
Listed:
- Courtney J Spoerer
- Tim C Kietzmann
- Johannes Mehrer
- Ian Charest
- Nikolaus Kriegeskorte
Abstract
Deep feedforward neural network models of vision dominate in both computational neuroscience and engineering. The primate visual system, by contrast, contains abundant recurrent connections. Recurrent signal flow enables recycling of limited computational resources over time, and so might boost the performance of a physically finite brain or model. Here we show: (1) Recurrent convolutional neural network models outperform feedforward convolutional models matched in their number of parameters in large-scale visual recognition tasks on natural images. (2) Setting a confidence threshold, at which recurrent computations terminate and a decision is made, enables flexible trading of speed for accuracy. At a given confidence threshold, the model expends more time and energy on images that are harder to recognise, without requiring additional parameters for deeper computations. (3) The recurrent model’s reaction time for an image predicts the human reaction time for the same image better than several parameter-matched and state-of-the-art feedforward models. (4) Across confidence thresholds, the recurrent model emulates the behaviour of feedforward control models in that it achieves the same accuracy at approximately the same computational cost (mean number of floating-point operations). However, the recurrent model can be run longer (higher confidence threshold) and then outperforms parameter-matched feedforward comparison models. These results suggest that recurrent connectivity, a hallmark of biological visual systems, may be essential for understanding the accuracy, flexibility, and dynamics of human visual recognition.Author summary: Deep neural networks provide the best current models of biological vision and achieve the highest performance in computer vision. Inspired by the primate brain, these models transform the image signals through a sequence of stages, leading to recognition. Unlike brains in which outputs of a given computation are fed back into the same computation, these models do not process signals recurrently. The ability to recycle limited neural resources by processing information recurrently could explain the accuracy and flexibility of biological visual systems, which computer vision systems cannot yet match. Here we report that recurrent processing can improve recognition performance compared to similarly complex feedforward networks. Recurrent processing also enabled models to behave more flexibly and trade off speed for accuracy. Like humans, the recurrent network models can compute longer when an object is hard to recognise, which boosts their accuracy. The model’s recognition times predicted human recognition times for the same images. The performance and flexibility of recurrent neural network models illustrates that modeling biological vision can help us improve computer vision.
Suggested Citation
Courtney J Spoerer & Tim C Kietzmann & Johannes Mehrer & Ian Charest & Nikolaus Kriegeskorte, 2020.
"Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision,"
PLOS Computational Biology, Public Library of Science, vol. 16(10), pages 1-27, October.
Handle:
RePEc:plo:pcbi00:1008215
DOI: 10.1371/journal.pcbi.1008215
Download full text from publisher
Citations
Citations are extracted by the
CitEc Project, subscribe to its
RSS feed for this item.
Cited by:
- Hui Wang & Ashutosh Sharma & Mohammad Shabaz, 2022.
"Research on digital media animation control technology based on recurrent neural network using speech technology,"
International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 13(1), pages 564-575, March.
- David Allen Axelrod, 2021.
"On the Obsolescence of Long-Run Rationality,"
RAIS Conference Proceedings 2021
0139, Research Association for Interdisciplinary Studies.
- Tian Zhu & Wei Zhu, 2022.
"Quantitative Trading through Random Perturbation Q-Network with Nonlinear Transaction Costs,"
Stats, MDPI, vol. 5(2), pages 1-15, June.
- Monika Graumann & Caterina Ciuffi & Kshitij Dwivedi & Gemma Roig & Radoslaw M. Cichy, 2022.
"The spatiotemporal neural dynamics of object location representations in the human brain,"
Nature Human Behaviour, Nature, vol. 6(6), pages 796-811, June.
Corrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pcbi00:1008215. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
We have no bibliographic references for this item. You can help adding them by using this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ploscompbiol (email available below). General contact details of provider: https://journals.plos.org/ploscompbiol/ .
Please note that corrections may take a couple of weeks to filter through
the various RePEc services.