IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v14y2023i1d10.1038_s41467-023-42488-9.html
   My bibliography  Save this article

Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction

Author

Listed:
  • Pei-Yu Huang

    (Harbin Institute of Technology (Shenzhen))

  • Bi-Yi Jiang

    (Southern University of Science and Technology
    The Hong Kong Polytechnic University)

  • Hong-Ji Chen

    (Harbin Institute of Technology (Shenzhen))

  • Jia-Yi Xu

    (Southern University of Science and Technology)

  • Kang Wang

    (Southeast University)

  • Cheng-Yi Zhu

    (Harbin Institute of Technology (Shenzhen))

  • Xin-Yan Hu

    (Southern University of Science and Technology)

  • Dong Li

    (Harbin Institute of Technology (Shenzhen))

  • Liang Zhen

    (Harbin Institute of Technology)

  • Fei-Chi Zhou

    (Southern University of Science and Technology)

  • Jing-Kai Qin

    (Harbin Institute of Technology (Shenzhen))

  • Cheng-Yan Xu

    (Harbin Institute of Technology (Shenzhen)
    Harbin Institute of Technology)

Abstract

Neuro-inspired vision systems hold great promise to address the growing demands of mass data processing for edge computing, a distributed framework that brings computation and data storage closer to the sources of data. In addition to the capability of static image sensing and processing, the hardware implementation of a neuro-inspired vision system also requires the fulfilment of detecting and recognizing moving targets. Here, we demonstrated a neuro-inspired optical sensor based on two-dimensional NbS2/MoS2 hybrid films, which featured remarkable photo-induced conductance plasticity and low electrical energy consumption. A neuro-inspired optical sensor array with 10 × 10 NbS2/MoS2 phototransistors enabled highly integrated functions of sensing, memory, and contrast enhancement capabilities for static images, which benefits convolutional neural network (CNN) with a high image recognition accuracy. More importantly, in-sensor trajectory registration of moving light spots was experimentally implemented such that the post-processing could yield a high restoration accuracy. Our neuro-inspired optical sensor array could provide a fascinating platform for the implementation of high-performance artificial vision systems.

Suggested Citation

  • Pei-Yu Huang & Bi-Yi Jiang & Hong-Ji Chen & Jia-Yi Xu & Kang Wang & Cheng-Yi Zhu & Xin-Yan Hu & Dong Li & Liang Zhen & Fei-Chi Zhou & Jing-Kai Qin & Cheng-Yan Xu, 2023. "Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
  • Handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-42488-9
    DOI: 10.1038/s41467-023-42488-9
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-023-42488-9
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-023-42488-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Zhongfang Zhang & Xiaolong Zhao & Xumeng Zhang & Xiaohu Hou & Xiaolan Ma & Shuangzhu Tang & Ying Zhang & Guangwei Xu & Qi Liu & Shibing Long, 2022. "In-sensor reservoir computing system for latent fingerprint recognition with deep ultraviolet photo-synapses and memristor array," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    2. Peng Yao & Huaqiang Wu & Bin Gao & Sukru Burc Eryilmaz & Xueyao Huang & Wenqiang Zhang & Qingtian Zhang & Ning Deng & Luping Shi & H.-S. Philip Wong & He Qian, 2017. "Face classification using electronic synapses," Nature Communications, Nature, vol. 8(1), pages 1-8, August.
    3. Kaushik Roy & Akhilesh Jaiswal & Priyadarshini Panda, 2019. "Towards spike-based machine intelligence with neuromorphic computing," Nature, Nature, vol. 575(7784), pages 607-617, November.
    4. Seongin Hong & Nicolò Zagni & Sooho Choo & Na Liu & Seungho Baek & Arindam Bala & Hocheon Yoo & Byung Ha Kang & Hyun Jae Kim & Hyung Joong Yun & Muhammad Ashraful Alam & Sunkook Kim, 2021. "Highly sensitive active pixel image sensor array driven by large-area bilayer MoS2 transistor circuitry," Nature Communications, Nature, vol. 12(1), pages 1-11, December.
    5. A. K. Geim & I. V. Grigorieva, 2013. "Van der Waals heterostructures," Nature, Nature, vol. 499(7459), pages 419-425, July.
    6. Xudong Ji & Bryan D. Paulsen & Gary K. K. Chik & Ruiheng Wu & Yuyang Yin & Paddy K. L. Chan & Jonathan Rivnay, 2021. "Mimicking associative learning using an ion-trapping non-volatile synaptic organic electrochemical transistor," Nature Communications, Nature, vol. 12(1), pages 1-12, December.
    7. Changsoon Choi & Juyoung Leem & Minsung Kim & Amir Taqieddin & Chullhee Cho & Kyoung Won Cho & Gil Ju Lee & Hyojin Seung & Hyung Jong Bae & Young Min Song & Taeghwan Hyeon & Narayana R. Aluru & SungWo, 2020. "Curved neuromorphic image sensor array using a MoS2-organic heterostructure inspired by the human visual recognition system," Nature Communications, Nature, vol. 11(1), pages 1-9, December.
    8. S. Hattar & R. J. Lucas & N. Mrosovsky & S. Thompson & R. H. Douglas & M. W. Hankins & J. Lem & M. Biel & F. Hofmann & R. G. Foster & K.-W. Yau, 2003. "Melanopsin and rod–cone photoreceptive systems account for all major accessory visual functions in mice," Nature, Nature, vol. 424(6944), pages 75-81, July.
    9. Lukas Mennel & Joanna Symonowicz & Stefan Wachter & Dmitry K. Polyushkin & Aday J. Molina-Mendoza & Thomas Mueller, 2020. "Ultrafast machine vision with 2D material neural network image sensors," Nature, Nature, vol. 579(7797), pages 62-66, March.
    10. Bence P. Ölveczky & Stephen A. Baccus & Markus Meister, 2003. "Segregation of object and background motion in the retina," Nature, Nature, vol. 423(6938), pages 401-408, May.
    11. Yuan Liu & Yu Huang & Xiangfeng Duan, 2019. "Van der Waals integration before and beyond two-dimensional materials," Nature, Nature, vol. 567(7748), pages 323-333, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yuyan Zhu & Yang Wang & Xingchen Pang & Yongbo Jiang & Xiaoxian Liu & Qing Li & Zhen Wang & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile 2D MoS2/black phosphorus heterojunction photodiodes in the near- to mid-infrared region," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    2. Ming Deng & Ziqing Li & Shiyuan Liu & Xiaosheng Fang & Limin Wu, 2024. "Wafer-scale integration of two-dimensional perovskite oxides towards motion recognition," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    3. Lei Xu & Junling Liu & Xinrui Guo & Shuo Liu & Xilin Lai & Jingyue Wang & Mengshi Yu & Zhengdao Xie & Hailin Peng & Xuming Zou & Xinran Wang & Ru Huang & Ming He, 2024. "Ultrasensitive dim-light neuromorphic vision sensing via momentum-conserved reconfigurable van der Waals heterostructure," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    4. Zhuiri Peng & Lei Tong & Wenhao Shi & Langlang Xu & Xinyu Huang & Zheng Li & Xiangxiang Yu & Xiaohan Meng & Xiao He & Shengjie Lv & Gaochen Yang & Hao Hao & Tian Jiang & Xiangshui Miao & Lei Ye, 2024. "Multifunctional human visual pathway-replicated hardware based on 2D materials," Nature Communications, Nature, vol. 15(1), pages 1-9, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tian Zhang & Xin Guo & Pan Wang & Xinyi Fan & Zichen Wang & Yan Tong & Decheng Wang & Limin Tong & Linjun Li, 2024. "High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    2. Lutao Li & Junjie Yao & Juntong Zhu & Yuan Chen & Chen Wang & Zhicheng Zhou & Guoxiang Zhao & Sihan Zhang & Ruonan Wang & Jiating Li & Xiangyi Wang & Zheng Lu & Lingbo Xiao & Qiang Zhang & Guifu Zou, 2023. "Colloid driven low supersaturation crystallization for atomically thin Bismuth halide perovskite," Nature Communications, Nature, vol. 14(1), pages 1-8, December.
    3. Pengshan Xie & Yunchao Xu & Jingwen Wang & Dengji Li & Yuxuan Zhang & Zixin Zeng & Boxiang Gao & Quan Quan & Bowen Li & You Meng & Weijun Wang & Yezhan Li & Yan Yan & Yi Shen & Jia Sun & Johnny C. Ho, 2024. "Birdlike broadband neuromorphic visual sensor arrays for fusion imaging," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    4. Chenhao Wang & Xinyi Xu & Xiaodong Pi & Mark D. Butala & Wen Huang & Lei Yin & Wenbing Peng & Munir Ali & Srikrishna Chanakya Bodepudi & Xvsheng Qiao & Yang Xu & Wei Sun & Deren Yang, 2022. "Neuromorphic device based on silicon nanosheets," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    5. Pengzhan Li & Mingzhen Zhang & Qingli Zhou & Qinghua Zhang & Donggang Xie & Ge Li & Zhuohui Liu & Zheng Wang & Erjia Guo & Meng He & Can Wang & Lin Gu & Guozhen Yang & Kuijuan Jin & Chen Ge, 2024. "Reconfigurable optoelectronic transistors for multimodal recognition," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    6. Xingchen Pang & Yang Wang & Yuyan Zhu & Zhenhan Zhang & Du Xiang & Xun Ge & Haoqi Wu & Yongbo Jiang & Zizheng Liu & Xiaoxian Liu & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile rippled-assisted optoelectronic array for all-day motion detection and recognition," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    7. Guangdong Zhou & Jie Li & Qunliang Song & Lidan Wang & Zhijun Ren & Bai Sun & Xiaofang Hu & Wenhua Wang & Gaobo Xu & Xiaodie Chen & Lan Cheng & Feichi Zhou & Shukai Duan, 2023. "Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    8. Sangyong Park & Dongyoung Lee & Juncheol Kang & Hojin Choi & Jin-Hong Park, 2023. "Laterally gated ferroelectric field effect transistor (LG-FeFET) using α-In2Se3 for stacked in-memory computing array," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    9. Zhuiri Peng & Lei Tong & Wenhao Shi & Langlang Xu & Xinyu Huang & Zheng Li & Xiangxiang Yu & Xiaohan Meng & Xiao He & Shengjie Lv & Gaochen Yang & Hao Hao & Tian Jiang & Xiangshui Miao & Lei Ye, 2024. "Multifunctional human visual pathway-replicated hardware based on 2D materials," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    10. Akhil Dodda & Nicholas Trainor & Joan. M. Redwing & Saptarshi Das, 2022. "All-in-one, bio-inspired, and low-power crypto engines for near-sensor security based on two-dimensional memtransistors," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    11. Doeon Lee & Minseong Park & Yongmin Baek & Byungjoon Bae & Junseok Heo & Kyusang Lee, 2022. "In-sensor image memorization and encoding via optical neurons for bio-stimulus domain reduction toward visual cognitive processing," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    12. Yuyan Zhu & Yang Wang & Xingchen Pang & Yongbo Jiang & Xiaoxian Liu & Qing Li & Zhen Wang & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile 2D MoS2/black phosphorus heterojunction photodiodes in the near- to mid-infrared region," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    13. Elyashiv Zangen & Shira Hadar & Christopher Lawrence & Mustafa Obeid & Hala Rasras & Ella Hanzin & Ori Aslan & Eyal Zur & Nadav Schulcz & Daniel Cohen-Hatab & Yona Samama & Sarah Nir & Yi Li & Irina D, 2024. "Prefrontal cortex neurons encode ambient light intensity differentially across regions and layers," Nature Communications, Nature, vol. 15(1), pages 1-19, December.
    14. John A. Gaynes & Samuel A. Budoff & Michael J. Grybko & Joshua B. Hunt & Alon Poleg-Polsky, 2022. "Classical center-surround receptive fields facilitate novel object detection in retinal bipolar cells," Nature Communications, Nature, vol. 13(1), pages 1-17, December.
    15. Pandey, Mayank & Deshmukh, Kalim & Raman, Akhila & Asok, Aparna & Appukuttan, Saritha & Suman, G.R., 2024. "Prospects of MXene and graphene for energy storage and conversion," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PB).
    16. Tao Guo & Shasha Li & Y. Norman Zhou & Wei D. Lu & Yong Yan & Yimin A. Wu, 2024. "Interspecies-chimera machine vision with polarimetry for real-time navigation and anti-glare pattern recognition," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    17. Benjamin Carey & Nils Kolja Wessling & Paul Steeger & Robert Schmidt & Steffen Michaelis de Vasconcellos & Rudolf Bratschitsch & Ashish Arora, 2024. "Giant Faraday rotation in atomically thin semiconductors," Nature Communications, Nature, vol. 15(1), pages 1-8, December.
    18. Yuri Saida & Thomas Gauthier & Hiroo Suzuki & Satoshi Ohmura & Ryo Shikata & Yui Iwasaki & Godai Noyama & Misaki Kishibuchi & Yuichiro Tanaka & Wataru Yajima & Nicolas Godin & Gaël Privault & Tomoharu, 2024. "Photoinduced dynamics during electronic transfer from narrow to wide bandgap layers in one-dimensional heterostructured materials," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    19. Ying Zhang & Ge-Qi Mao & Xiaolong Zhao & Yu Li & Meiyun Zhang & Zuheng Wu & Wei Wu & Huajun Sun & Yizhong Guo & Lihua Wang & Xumeng Zhang & Qi Liu & Hangbing Lv & Kan-Hao Xue & Guangwei Xu & Xiangshui, 2021. "Evolution of the conductive filament system in HfO2-based memristors observed by direct atomic-scale imaging," Nature Communications, Nature, vol. 12(1), pages 1-10, December.
    20. Ying-Xin Ma & Xue-Dong Wang, 2024. "Directional self-assembly of organic vertically superposed nanowires," Nature Communications, Nature, vol. 15(1), pages 1-9, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-42488-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.