IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-50488-6.html
   My bibliography  Save this article

Adaptative machine vision with microsecond-level accurate perception beyond human retina

Author

Listed:
  • Ling Li

    (South China Normal University)

  • Shasha Li

    (Chaohu University)

  • Wenhai Wang

    (South China Normal University)

  • Jielian Zhang

    (South China Normal University)

  • Yiming Sun

    (South China Normal University)

  • Qunrui Deng

    (South China Normal University)

  • Tao Zheng

    (South China Normal University)

  • Jianting Lu

    (China Electronic Product Reliability and Environmental Testing Research Institute)

  • Wei Gao

    (South China Normal University)

  • Mengmeng Yang

    (South China Normal University)

  • Hanyu Wang

    (South China Normal University)

  • Yuan Pan

    (South China Normal University)

  • Xueting Liu

    (South China Normal University)

  • Yani Yang

    (South China Normal University)

  • Jingbo Li

    (College of Optical Science and Engineering
    Guangdong Provincial Key Laboratory of Chip and Integration Technology)

  • Nengjie Huo

    (South China Normal University
    Guangdong Provincial Key Laboratory of Chip and Integration Technology)

Abstract

Visual adaptive devices have potential to simplify circuits and algorithms in machine vision systems to adapt and perceive images with varying brightness levels, which is however limited by sluggish adaptation process. Here, the avalanche tuning as feedforward inhibition in bionic two-dimensional (2D) transistor is proposed for fast and high-frequency visual adaptation behavior with microsecond-level accurate perception, the adaptation speed is over 104 times faster than that of human retina and reported bionic sensors. As light intensity changes, the bionic transistor spontaneously switches between avalanche and photoconductive effect, varying responsivity in both magnitude and sign (from 7.6 × 104 to −1 × 103 A/W), thereby achieving ultra-fast scotopic and photopic adaptation process of 108 and 268 μs, respectively. By further combining convolutional neural networks with avalanche-tuned bionic transistor, an adaptative machine vision is achieved with remarkable microsecond-level rapid adaptation capabilities and robust image recognition with over 98% precision in both dim and bright conditions.

Suggested Citation

  • Ling Li & Shasha Li & Wenhai Wang & Jielian Zhang & Yiming Sun & Qunrui Deng & Tao Zheng & Jianting Lu & Wei Gao & Mengmeng Yang & Hanyu Wang & Yuan Pan & Xueting Liu & Yani Yang & Jingbo Li & Nengjie, 2024. "Adaptative machine vision with microsecond-level accurate perception beyond human retina," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-50488-6
    DOI: 10.1038/s41467-024-50488-6
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-50488-6
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-50488-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Lukas Mennel & Joanna Symonowicz & Stefan Wachter & Dmitry K. Polyushkin & Aday J. Molina-Mendoza & Thomas Mueller, 2020. "Ultrafast machine vision with 2D material neural network image sensors," Nature, Nature, vol. 579(7797), pages 62-66, March.
    2. Fan Pan & Luo Sun & David B. Kardian & Katharine A. Whartenby & Drew M. Pardoll & Jun O. Liu, 2007. "Feedback inhibition of calcineurin and Ras by a dual inhibitory protein Carabin," Nature, Nature, vol. 445(7126), pages 433-436, January.
    3. Brent Doiron & Maurice J. Chacron & Leonard Maler & André Longtin & Joseph Bastian, 2003. "Inhibitory feedback required for network oscillatory responses to communication but not prey stimuli," Nature, Nature, vol. 421(6922), pages 539-543, January.
    4. Gwendal Le Masson & Sylvie Renaud-Le Masson & Damien Debay & Thierry Bal, 2002. "Feedback inhibition controls spike transfer in hybrid thalamic circuits," Nature, Nature, vol. 417(6891), pages 854-858, June.
    5. Huayu Ding & Robert G. Smith & Alon Poleg-Polsky & Jeffrey S. Diamond & Kevin L. Briggman, 2016. "Species-specific wiring for direction selectivity in the mammalian retina," Nature, Nature, vol. 535(7610), pages 105-110, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuyan Zhu & Yang Wang & Xingchen Pang & Yongbo Jiang & Xiaoxian Liu & Qing Li & Zhen Wang & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile 2D MoS2/black phosphorus heterojunction photodiodes in the near- to mid-infrared region," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    2. John A. Gaynes & Samuel A. Budoff & Michael J. Grybko & Joshua B. Hunt & Alon Poleg-Polsky, 2022. "Classical center-surround receptive fields facilitate novel object detection in retinal bipolar cells," Nature Communications, Nature, vol. 13(1), pages 1-17, December.
    3. Tao Guo & Shasha Li & Y. Norman Zhou & Wei D. Lu & Yong Yan & Yimin A. Wu, 2024. "Interspecies-chimera machine vision with polarimetry for real-time navigation and anti-glare pattern recognition," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    4. Yan Sun & Shuting Xu & Zheqi Xu & Jiamin Tian & Mengmeng Bai & Zhiying Qi & Yue Niu & Hein Htet Aung & Xiaolu Xiong & Junfeng Han & Cuicui Lu & Jianbo Yin & Sheng Wang & Qing Chen & Reshef Tenne & All, 2022. "Mesoscopic sliding ferroelectricity enabled photovoltaic random access memory for material-level artificial vision system," Nature Communications, Nature, vol. 13(1), pages 1-8, December.
    5. Pei-Yu Huang & Bi-Yi Jiang & Hong-Ji Chen & Jia-Yi Xu & Kang Wang & Cheng-Yi Zhu & Xin-Yan Hu & Dong Li & Liang Zhen & Fei-Chi Zhou & Jing-Kai Qin & Cheng-Yan Xu, 2023. "Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    6. Junxiong Guo & Shuyi Gu & Lin Lin & Yu Liu & Ji Cai & Hongyi Cai & Yu Tian & Yuelin Zhang & Qinghua Zhang & Ze Liu & Yafei Zhang & Xiaosheng Zhang & Yuan Lin & Wen Huang & Lin Gu & Jinxing Zhang, 2024. "Type-printable photodetector arrays for multichannel meta-infrared imaging," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    7. Xinxin Gao & Ze Gu & Qian Ma & Bao Jie Chen & Kam-Man Shum & Wen Yi Cui & Jian Wei You & Tie Jun Cui & Chi Hou Chan, 2024. "Terahertz spoof plasmonic neural network for diffractive information recognition and processing," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    8. Ke Yang & Yanghao Wang & Pek Jun Tiw & Chaoming Wang & Xiaolong Zou & Rui Yuan & Chang Liu & Ge Li & Chen Ge & Si Wu & Teng Zhang & Ru Huang & Yuchao Yang, 2024. "High-order sensory processing nanocircuit based on coupled VO2 oscillators," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    9. Yeon Jin Kim & Beth B. Peterson & Joanna D. Crook & Hannah R. Joo & Jiajia Wu & Christian Puller & Farrel R. Robinson & Paul D. Gamlin & King-Wai Yau & Felix Viana & John B. Troy & Robert G. Smith & O, 2022. "Origins of direction selectivity in the primate retina," Nature Communications, Nature, vol. 13(1), pages 1-20, December.
    10. Xinyu Chen & Yufeng Xie & Yaochen Sheng & Hongwei Tang & Zeming Wang & Yu Wang & Yin Wang & Fuyou Liao & Jingyi Ma & Xiaojiao Guo & Ling Tong & Hanqi Liu & Hao Liu & Tianxiang Wu & Jiaxin Cao & Sitong, 2021. "Wafer-scale functional circuits based on two dimensional semiconductors with fabrication optimized by machine learning," Nature Communications, Nature, vol. 12(1), pages 1-8, December.
    11. Mitsumasa Nakajima & Katsuma Inoue & Kenji Tanaka & Yasuo Kuniyoshi & Toshikazu Hashimoto & Kohei Nakajima, 2022. "Physical deep learning with biologically inspired training method: gradient-free approach for physical hardware," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    12. Xingchen Pang & Yang Wang & Yuyan Zhu & Zhenhan Zhang & Du Xiang & Xun Ge & Haoqi Wu & Yongbo Jiang & Zizheng Liu & Xiaoxian Liu & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile rippled-assisted optoelectronic array for all-day motion detection and recognition," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    13. Carles Bosch & Tobias Ackels & Alexandra Pacureanu & Yuxin Zhang & Christopher J. Peddie & Manuel Berning & Norman Rzepka & Marie-Christine Zdora & Isabell Whiteley & Malte Storm & Anne Bonnin & Chris, 2022. "Functional and multiscale 3D structural investigation of brain tissue through correlative in vivo physiology, synchrotron microtomography and volume electron microscopy," Nature Communications, Nature, vol. 13(1), pages 1-16, December.
    14. Haihui Lan & Luyang Wang & Runze He & Shuyi Huang & Jinqiu Yu & Jinming Guo & Jingrui Luo & Yiling Li & Jinyang Zhang & Jiaxin Lin & Shunping Zhang & Mengqi Zeng & Lei Fu, 2023. "2D quasi-layered material with domino structure," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    15. David Swygart & Wan-Qing Yu & Shunsuke Takeuchi & Rachel O. L. Wong & Gregory W. Schwartz, 2024. "A presynaptic source drives differing levels of surround suppression in two mouse retinal ganglion cell types," Nature Communications, Nature, vol. 15(1), pages 1-20, December.
    16. Xiaopeng Feng & Chenglong Li & Jinmei Song & Yuhong He & Wei Qu & Weijun Li & Keke Guo & Lulu Liu & Bai Yang & Haotong Wei, 2024. "Differential perovskite hemispherical photodetector for intelligent imaging and location tracking," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    17. Héctor Acarón Ledesma & Jennifer Ding & Swen Oosterboer & Xiaolin Huang & Qiang Chen & Sui Wang & Michael Z. Lin & Wei Wei, 2024. "Dendritic mGluR2 and perisomatic Kv3 signaling regulate dendritic computation of mouse starburst amacrine cells," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    18. Guangdong Zhou & Jie Li & Qunliang Song & Lidan Wang & Zhijun Ren & Bai Sun & Xiaofang Hu & Wenhua Wang & Gaobo Xu & Xiaodie Chen & Lan Cheng & Feichi Zhou & Shukai Duan, 2023. "Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    19. Boyuan Cui & Zhen Fan & Wenjie Li & Yihong Chen & Shuai Dong & Zhengwei Tan & Shengliang Cheng & Bobo Tian & Ruiqiang Tao & Guo Tian & Deyang Chen & Zhipeng Hou & Minghui Qin & Min Zeng & Xubing Lu & , 2022. "Ferroelectric photosensor network: an advanced hardware solution to real-time machine vision," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    20. Tian Zhang & Xin Guo & Pan Wang & Xinyi Fan & Zichen Wang & Yan Tong & Decheng Wang & Limin Tong & Linjun Li, 2024. "High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network," Nature Communications, Nature, vol. 15(1), pages 1-10, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-50488-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.