IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-50488-6.html
   My bibliography  Save this article

Adaptative machine vision with microsecond-level accurate perception beyond human retina

Author

Listed:
  • Ling Li

    (South China Normal University)

  • Shasha Li

    (Chaohu University)

  • Wenhai Wang

    (South China Normal University)

  • Jielian Zhang

    (South China Normal University)

  • Yiming Sun

    (South China Normal University)

  • Qunrui Deng

    (South China Normal University)

  • Tao Zheng

    (South China Normal University)

  • Jianting Lu

    (China Electronic Product Reliability and Environmental Testing Research Institute)

  • Wei Gao

    (South China Normal University)

  • Mengmeng Yang

    (South China Normal University)

  • Hanyu Wang

    (South China Normal University)

  • Yuan Pan

    (South China Normal University)

  • Xueting Liu

    (South China Normal University)

  • Yani Yang

    (South China Normal University)

  • Jingbo Li

    (College of Optical Science and Engineering
    Guangdong Provincial Key Laboratory of Chip and Integration Technology)

  • Nengjie Huo

    (South China Normal University
    Guangdong Provincial Key Laboratory of Chip and Integration Technology)

Abstract

Visual adaptive devices have potential to simplify circuits and algorithms in machine vision systems to adapt and perceive images with varying brightness levels, which is however limited by sluggish adaptation process. Here, the avalanche tuning as feedforward inhibition in bionic two-dimensional (2D) transistor is proposed for fast and high-frequency visual adaptation behavior with microsecond-level accurate perception, the adaptation speed is over 104 times faster than that of human retina and reported bionic sensors. As light intensity changes, the bionic transistor spontaneously switches between avalanche and photoconductive effect, varying responsivity in both magnitude and sign (from 7.6 × 104 to −1 × 103 A/W), thereby achieving ultra-fast scotopic and photopic adaptation process of 108 and 268 μs, respectively. By further combining convolutional neural networks with avalanche-tuned bionic transistor, an adaptative machine vision is achieved with remarkable microsecond-level rapid adaptation capabilities and robust image recognition with over 98% precision in both dim and bright conditions.

Suggested Citation

  • Ling Li & Shasha Li & Wenhai Wang & Jielian Zhang & Yiming Sun & Qunrui Deng & Tao Zheng & Jianting Lu & Wei Gao & Mengmeng Yang & Hanyu Wang & Yuan Pan & Xueting Liu & Yani Yang & Jingbo Li & Nengjie, 2024. "Adaptative machine vision with microsecond-level accurate perception beyond human retina," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-50488-6
    DOI: 10.1038/s41467-024-50488-6
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-50488-6
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-50488-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Fan Pan & Luo Sun & David B. Kardian & Katharine A. Whartenby & Drew M. Pardoll & Jun O. Liu, 2007. "Feedback inhibition of calcineurin and Ras by a dual inhibitory protein Carabin," Nature, Nature, vol. 445(7126), pages 433-436, January.
    2. Huayu Ding & Robert G. Smith & Alon Poleg-Polsky & Jeffrey S. Diamond & Kevin L. Briggman, 2016. "Species-specific wiring for direction selectivity in the mammalian retina," Nature, Nature, vol. 535(7610), pages 105-110, July.
    3. Gwendal Le Masson & Sylvie Renaud-Le Masson & Damien Debay & Thierry Bal, 2002. "Feedback inhibition controls spike transfer in hybrid thalamic circuits," Nature, Nature, vol. 417(6891), pages 854-858, June.
    4. Lukas Mennel & Joanna Symonowicz & Stefan Wachter & Dmitry K. Polyushkin & Aday J. Molina-Mendoza & Thomas Mueller, 2020. "Ultrafast machine vision with 2D material neural network image sensors," Nature, Nature, vol. 579(7797), pages 62-66, March.
    5. Brent Doiron & Maurice J. Chacron & Leonard Maler & André Longtin & Joseph Bastian, 2003. "Inhibitory feedback required for network oscillatory responses to communication but not prey stimuli," Nature, Nature, vol. 421(6922), pages 539-543, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhuiri Peng & Lei Tong & Wenhao Shi & Langlang Xu & Xinyu Huang & Zheng Li & Xiangxiang Yu & Xiaohan Meng & Xiao He & Shengjie Lv & Gaochen Yang & Hao Hao & Tian Jiang & Xiangshui Miao & Lei Ye, 2024. "Multifunctional human visual pathway-replicated hardware based on 2D materials," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    2. Yuyan Zhu & Yang Wang & Xingchen Pang & Yongbo Jiang & Xiaoxian Liu & Qing Li & Zhen Wang & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile 2D MoS2/black phosphorus heterojunction photodiodes in the near- to mid-infrared region," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    3. Xun Han & Juan Tao & Yegang Liang & Feng Guo & Zhangsheng Xu & Wenqiang Wu & Jiahui Tong & Mengxiao Chen & Caofeng Pan & Jianhua Hao, 2024. "Ultraweak light-modulated heterostructure with bidirectional photoresponse for static and dynamic image perception," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    4. Zhixin Yao & Huifeng Tian & U. Sasaki & Huacong Sun & Jingyi Hu & Guodong Xue & Ye Seul Jung & Ruijie Li & Zhenjiang Li & PeiChi Liao & Yihan Wang & Lina Yang Zhang & Ge Yin & Xuanyu Zhang & Yijie Luo, 2025. "Transferrable, wet-chemistry-derived high-k amorphous metal oxide dielectrics for two-dimensional electronic devices," Nature Communications, Nature, vol. 16(1), pages 1-11, December.
    5. Lei Tong & Yali Bi & Yilun Wang & Kai Peng & Xinyu Huang & Wei Ju & Zhuiri Peng & Zheng Li & Langlang Xu & Runfeng Lin & Xiangxiang Yu & Wenhao Shi & Hui Yu & Huajun Sun & Kanhao Xue & Qiang He & Ming, 2024. "Programmable nonlinear optical neuromorphic computing with bare 2D material MoS2," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    6. Ge Li & Donggang Xie & Qinghua Zhang & Mingzhen Zhang & Zhuohui Liu & Zheng Wang & Jiahui Xie & Erjia Guo & Meng He & Can Wang & Lin Gu & Guozhen Yang & Kuijuan Jin & Chen Ge, 2025. "Interface-engineered non-volatile visible-blind photodetector for in-sensor computing," Nature Communications, Nature, vol. 16(1), pages 1-10, December.
    7. Ashok Litwin-Kumar & Maurice J Chacron & Brent Doiron, 2012. "The Spatial Structure of Stimuli Shapes the Timescale of Correlations in Population Spiking Activity," PLOS Computational Biology, Public Library of Science, vol. 8(9), pages 1-15, September.
    8. Yeon Jin Kim & Beth B. Peterson & Joanna D. Crook & Hannah R. Joo & Jiajia Wu & Christian Puller & Farrel R. Robinson & Paul D. Gamlin & King-Wai Yau & Felix Viana & John B. Troy & Robert G. Smith & O, 2022. "Origins of direction selectivity in the primate retina," Nature Communications, Nature, vol. 13(1), pages 1-20, December.
    9. Xinyu Chen & Yufeng Xie & Yaochen Sheng & Hongwei Tang & Zeming Wang & Yu Wang & Yin Wang & Fuyou Liao & Jingyi Ma & Xiaojiao Guo & Ling Tong & Hanqi Liu & Hao Liu & Tianxiang Wu & Jiaxin Cao & Sitong, 2021. "Wafer-scale functional circuits based on two dimensional semiconductors with fabrication optimized by machine learning," Nature Communications, Nature, vol. 12(1), pages 1-8, December.
    10. Xingchen Pang & Yang Wang & Yuyan Zhu & Zhenhan Zhang & Du Xiang & Xun Ge & Haoqi Wu & Yongbo Jiang & Zizheng Liu & Xiaoxian Liu & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile rippled-assisted optoelectronic array for all-day motion detection and recognition," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    11. Carles Bosch & Tobias Ackels & Alexandra Pacureanu & Yuxin Zhang & Christopher J. Peddie & Manuel Berning & Norman Rzepka & Marie-Christine Zdora & Isabell Whiteley & Malte Storm & Anne Bonnin & Chris, 2022. "Functional and multiscale 3D structural investigation of brain tissue through correlative in vivo physiology, synchrotron microtomography and volume electron microscopy," Nature Communications, Nature, vol. 13(1), pages 1-16, December.
    12. Pablo Chamorro & Carlos Muñiz & Rafael Levi & David Arroyo & Francisco B Rodríguez & Pablo Varona, 2012. "Generalization of the Dynamic Clamp Concept in Neurophysiology and Behavior," PLOS ONE, Public Library of Science, vol. 7(7), pages 1-10, July.
    13. Bo Tong & Jiajun Xu & Jinhong Du & Peitao Liu & Tianming Du & Qiang Wang & Langjun Li & Yuning Wei & Jiangxu Li & Jinhua Liang & Chi Liu & Zhibo Liu & Chen Li & Lai-Peng Ma & Yang Chai & Wencai Ren, 2025. "2D (NH4)BiI3 enables non-volatile optoelectronic memories for machine learning," Nature Communications, Nature, vol. 16(1), pages 1-11, December.
    14. Haihui Lan & Luyang Wang & Runze He & Shuyi Huang & Jinqiu Yu & Jinming Guo & Jingrui Luo & Yiling Li & Jinyang Zhang & Jiaxin Lin & Shunping Zhang & Mengqi Zeng & Lei Fu, 2023. "2D quasi-layered material with domino structure," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    15. David Swygart & Wan-Qing Yu & Shunsuke Takeuchi & Rachel O. L. Wong & Gregory W. Schwartz, 2024. "A presynaptic source drives differing levels of surround suppression in two mouse retinal ganglion cell types," Nature Communications, Nature, vol. 15(1), pages 1-20, December.
    16. Yue Gong & Ruihuan Duan & Yi Hu & Yao Wu & Song Zhu & Xingli Wang & Qijie Wang & Shu Ping Lau & Zheng Liu & Beng Kang Tay, 2025. "Reconfigurable and nonvolatile ferroelectric bulk photovoltaics based on 3R-WS2 for machine vision," Nature Communications, Nature, vol. 16(1), pages 1-12, December.
    17. Tian Zhang & Xin Guo & Pan Wang & Xinyi Fan & Zichen Wang & Yan Tong & Decheng Wang & Limin Tong & Linjun Li, 2024. "High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    18. Hao Jiang & Yinzhu Chen & Wenyu Guo & Yan Zhang & Rigui Zhou & Mile Gu & Fan Zhong & Zhenhua Ni & Junpeng Lu & Cheng-Wei Qiu & Weibo Gao, 2024. "Metasurface-enabled broadband multidimensional photodetectors," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    19. Niklas Brake & Flavie Duc & Alexander Rokos & Francis Arseneau & Shiva Shahiri & Anmar Khadra & Gilles Plourde, 2024. "A neurophysiological basis for aperiodic EEG and the background spectral trend," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    20. Tyler R. Sizemore & Julius Jonaitis & Andrew M. Dacks, 2023. "Heterogeneous receptor expression underlies non-uniform peptidergic modulation of olfaction in Drosophila," Nature Communications, Nature, vol. 14(1), pages 1-22, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-50488-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.