IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v14y2023i1d10.1038_s41467-023-42488-9.html
   My bibliography  Save this article

Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction

Author

Listed:
  • Pei-Yu Huang

    (Harbin Institute of Technology (Shenzhen))

  • Bi-Yi Jiang

    (Southern University of Science and Technology
    The Hong Kong Polytechnic University)

  • Hong-Ji Chen

    (Harbin Institute of Technology (Shenzhen))

  • Jia-Yi Xu

    (Southern University of Science and Technology)

  • Kang Wang

    (Southeast University)

  • Cheng-Yi Zhu

    (Harbin Institute of Technology (Shenzhen))

  • Xin-Yan Hu

    (Southern University of Science and Technology)

  • Dong Li

    (Harbin Institute of Technology (Shenzhen))

  • Liang Zhen

    (Harbin Institute of Technology)

  • Fei-Chi Zhou

    (Southern University of Science and Technology)

  • Jing-Kai Qin

    (Harbin Institute of Technology (Shenzhen))

  • Cheng-Yan Xu

    (Harbin Institute of Technology (Shenzhen)
    Harbin Institute of Technology)

Abstract

Neuro-inspired vision systems hold great promise to address the growing demands of mass data processing for edge computing, a distributed framework that brings computation and data storage closer to the sources of data. In addition to the capability of static image sensing and processing, the hardware implementation of a neuro-inspired vision system also requires the fulfilment of detecting and recognizing moving targets. Here, we demonstrated a neuro-inspired optical sensor based on two-dimensional NbS2/MoS2 hybrid films, which featured remarkable photo-induced conductance plasticity and low electrical energy consumption. A neuro-inspired optical sensor array with 10 × 10 NbS2/MoS2 phototransistors enabled highly integrated functions of sensing, memory, and contrast enhancement capabilities for static images, which benefits convolutional neural network (CNN) with a high image recognition accuracy. More importantly, in-sensor trajectory registration of moving light spots was experimentally implemented such that the post-processing could yield a high restoration accuracy. Our neuro-inspired optical sensor array could provide a fascinating platform for the implementation of high-performance artificial vision systems.

Suggested Citation

  • Pei-Yu Huang & Bi-Yi Jiang & Hong-Ji Chen & Jia-Yi Xu & Kang Wang & Cheng-Yi Zhu & Xin-Yan Hu & Dong Li & Liang Zhen & Fei-Chi Zhou & Jing-Kai Qin & Cheng-Yan Xu, 2023. "Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    • Handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-42488-9
    DOI: 10.1038/s41467-023-42488-9
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-023-42488-9
    File Function: Abstract
    Download Restriction: no
    File URL: https://libkey.io/10.1038/s41467-023-42488-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Zhongfang Zhang & Xiaolong Zhao & Xumeng Zhang & Xiaohu Hou & Xiaolan Ma & Shuangzhu Tang & Ying Zhang & Guangwei Xu & Qi Liu & Shibing Long, 2022. "In-sensor reservoir computing system for latent fingerprint recognition with deep ultraviolet photo-synapses and memristor array," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    2. Peng Yao & Huaqiang Wu & Bin Gao & Sukru Burc Eryilmaz & Xueyao Huang & Wenqiang Zhang & Qingtian Zhang & Ning Deng & Luping Shi & H.-S. Philip Wong & He Qian, 2017. "Face classification using electronic synapses," Nature Communications, Nature, vol. 8(1), pages 1-8, August.
    3. Seongin Hong & Nicolò Zagni & Sooho Choo & Na Liu & Seungho Baek & Arindam Bala & Hocheon Yoo & Byung Ha Kang & Hyun Jae Kim & Hyung Joong Yun & Muhammad Ashraful Alam & Sunkook Kim, 2021. "Highly sensitive active pixel image sensor array driven by large-area bilayer MoS2 transistor circuitry," Nature Communications, Nature, vol. 12(1), pages 1-11, December.
    4. Changsoon Choi & Juyoung Leem & Minsung Kim & Amir Taqieddin & Chullhee Cho & Kyoung Won Cho & Gil Ju Lee & Hyojin Seung & Hyung Jong Bae & Young Min Song & Taeghwan Hyeon & Narayana R. Aluru & SungWo, 2020. "Curved neuromorphic image sensor array using a MoS2-organic heterostructure inspired by the human visual recognition system," Nature Communications, Nature, vol. 11(1), pages 1-9, December.
    5. Lukas Mennel & Joanna Symonowicz & Stefan Wachter & Dmitry K. Polyushkin & Aday J. Molina-Mendoza & Thomas Mueller, 2020. "Ultrafast machine vision with 2D material neural network image sensors," Nature, Nature, vol. 579(7797), pages 62-66, March.
    6. Bence P. Ölveczky & Stephen A. Baccus & Markus Meister, 2003. "Segregation of object and background motion in the retina," Nature, Nature, vol. 423(6938), pages 401-408, May.
    7. Yuan Liu & Yu Huang & Xiangfeng Duan, 2019. "Van der Waals integration before and beyond two-dimensional materials," Nature, Nature, vol. 567(7748), pages 323-333, March.
    8. Kaushik Roy & Akhilesh Jaiswal & Priyadarshini Panda, 2019. "Towards spike-based machine intelligence with neuromorphic computing," Nature, Nature, vol. 575(7784), pages 607-617, November.
    9. A. K. Geim & I. V. Grigorieva, 2013. "Van der Waals heterostructures," Nature, Nature, vol. 499(7459), pages 419-425, July.
    10. Xudong Ji & Bryan D. Paulsen & Gary K. K. Chik & Ruiheng Wu & Yuyang Yin & Paddy K. L. Chan & Jonathan Rivnay, 2021. "Mimicking associative learning using an ion-trapping non-volatile synaptic organic electrochemical transistor," Nature Communications, Nature, vol. 12(1), pages 1-12, December.
    11. S. Hattar & R. J. Lucas & N. Mrosovsky & S. Thompson & R. H. Douglas & M. W. Hankins & J. Lem & M. Biel & F. Hofmann & R. G. Foster & K.-W. Yau, 2003. "Melanopsin and rod–cone photoreceptive systems account for all major accessory visual functions in mice," Nature, Nature, vol. 424(6944), pages 75-81, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yuyan Zhu & Yang Wang & Xingchen Pang & Yongbo Jiang & Xiaoxian Liu & Qing Li & Zhen Wang & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile 2D MoS2/black phosphorus heterojunction photodiodes in the near- to mid-infrared region," Nature Communications, Nature, vol. 15(1), pages 1-10, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tian Zhang & Xin Guo & Pan Wang & Xinyi Fan & Zichen Wang & Yan Tong & Decheng Wang & Limin Tong & Linjun Li, 2024. "High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    2. Xingchen Pang & Yang Wang & Yuyan Zhu & Zhenhan Zhang & Du Xiang & Xun Ge & Haoqi Wu & Yongbo Jiang & Zizheng Liu & Xiaoxian Liu & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile rippled-assisted optoelectronic array for all-day motion detection and recognition," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    3. Sangyong Park & Dongyoung Lee & Juncheol Kang & Hojin Choi & Jin-Hong Park, 2023. "Laterally gated ferroelectric field effect transistor (LG-FeFET) using α-In2Se3 for stacked in-memory computing array," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    4. Doeon Lee & Minseong Park & Yongmin Baek & Byungjoon Bae & Junseok Heo & Kyusang Lee, 2022. "In-sensor image memorization and encoding via optical neurons for bio-stimulus domain reduction toward visual cognitive processing," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    5. Lutao Li & Junjie Yao & Juntong Zhu & Yuan Chen & Chen Wang & Zhicheng Zhou & Guoxiang Zhao & Sihan Zhang & Ruonan Wang & Jiating Li & Xiangyi Wang & Zheng Lu & Lingbo Xiao & Qiang Zhang & Guifu Zou, 2023. "Colloid driven low supersaturation crystallization for atomically thin Bismuth halide perovskite," Nature Communications, Nature, vol. 14(1), pages 1-8, December.
    6. Chenhao Wang & Xinyi Xu & Xiaodong Pi & Mark D. Butala & Wen Huang & Lei Yin & Wenbing Peng & Munir Ali & Srikrishna Chanakya Bodepudi & Xvsheng Qiao & Yang Xu & Wei Sun & Deren Yang, 2022. "Neuromorphic device based on silicon nanosheets," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    7. Pengzhan Li & Mingzhen Zhang & Qingli Zhou & Qinghua Zhang & Donggang Xie & Ge Li & Zhuohui Liu & Zheng Wang & Erjia Guo & Meng He & Can Wang & Lin Gu & Guozhen Yang & Kuijuan Jin & Chen Ge, 2024. "Reconfigurable optoelectronic transistors for multimodal recognition," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    8. Guangdong Zhou & Jie Li & Qunliang Song & Lidan Wang & Zhijun Ren & Bai Sun & Xiaofang Hu & Wenhua Wang & Gaobo Xu & Xiaodie Chen & Lan Cheng & Feichi Zhou & Shukai Duan, 2023. "Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    9. Akhil Dodda & Nicholas Trainor & Joan. M. Redwing & Saptarshi Das, 2022. "All-in-one, bio-inspired, and low-power crypto engines for near-sensor security based on two-dimensional memtransistors," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    10. Yuyan Zhu & Yang Wang & Xingchen Pang & Yongbo Jiang & Xiaoxian Liu & Qing Li & Zhen Wang & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile 2D MoS2/black phosphorus heterojunction photodiodes in the near- to mid-infrared region," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    11. Pandey, Mayank & Deshmukh, Kalim & Raman, Akhila & Asok, Aparna & Appukuttan, Saritha & Suman, G.R., 2024. "Prospects of MXene and graphene for energy storage and conversion," Renewable and Sustainable Energy Reviews, Elsevier, vol. 189(PB).
    12. Benjamin Carey & Nils Kolja Wessling & Paul Steeger & Robert Schmidt & Steffen Michaelis de Vasconcellos & Rudolf Bratschitsch & Ashish Arora, 2024. "Giant Faraday rotation in atomically thin semiconductors," Nature Communications, Nature, vol. 15(1), pages 1-8, December.
    13. Choi, Woo Sik & Jang, Jun Tae & Kim, Donguk & Yang, Tae Jun & Kim, Changwook & Kim, Hyungjin & Kim, Dae Hwan, 2022. "Influence of Al2O3 layer on InGaZnO memristor crossbar array for neuromorphic applications," Chaos, Solitons & Fractals, Elsevier, vol. 156(C).
    14. Zhenghao Long & Xiao Qiu & Chak Lam Jonathan Chan & Zhibo Sun & Zhengnan Yuan & Swapnadeep Poddar & Yuting Zhang & Yucheng Ding & Leilei Gu & Yu Zhou & Wenying Tang & Abhishek Kumar Srivastava & Cunji, 2023. "A neuromorphic bionic eye with filter-free color vision using hemispherical perovskite nanowire array retina," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    15. Fanfan Li & Dingwei Li & Chuanqing Wang & Guolei Liu & Rui Wang & Huihui Ren & Yingjie Tang & Yan Wang & Yitong Chen & Kun Liang & Qi Huang & Mohamad Sawan & Min Qiu & Hong Wang & Bowen Zhu, 2024. "An artificial visual neuron with multiplexed rate and time-to-first-spike coding," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    16. Eli Gerber & Steven B. Torrisi & Sara Shabani & Eric Seewald & Jordan Pack & Jennifer E. Hoffman & Cory R. Dean & Abhay N. Pasupathy & Eun-Ah Kim, 2023. "High-throughput ab initio design of atomic interfaces using InterMatch," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    17. Dong Gue Roe & Dong Hae Ho & Yoon Young Choi & Young Jin Choi & Seongchan Kim & Sae Byeok Jo & Moon Sung Kang & Jong-Hyun Ahn & Jeong Ho Cho, 2023. "Humanlike spontaneous motion coordination of robotic fingers through spatial multi-input spike signal multiplexing," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    18. Padinhare Cholakkal Harikesh & Chi-Yuan Yang & Deyu Tu & Jennifer Y. Gerasimov & Abdul Manan Dar & Adam Armada-Moreira & Matteo Massetti & Renee Kroon & David Bliman & Roger Olsson & Eleni Stavrinidou, 2022. "Organic electrochemical neurons and synapses with ion mediated spiking," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    19. Bohayra Mortazavi & Timon Rabczuk, 2018. "Boron Monochalcogenides; Stable and Strong Two-Dimensional Wide Band-Gap Semiconductors," Energies, MDPI, vol. 11(6), pages 1-10, June.
    20. Yeonghun Lee & Yaoqiao Hu & Xiuyao Lang & Dongwook Kim & Kejun Li & Yuan Ping & Kai-Mei C. Fu & Kyeongjae Cho, 2022. "Spin-defect qubits in two-dimensional transition metal dichalcogenides operating at telecom wavelengths," Nature Communications, Nature, vol. 13(1), pages 1-10, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:14:y:2023:i:1:d:10.1038_s41467-023-42488-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.