IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v13y2022i1d10.1038_s41467-022-29364-8.html
   My bibliography  Save this article

Ferroelectric photosensor network: an advanced hardware solution to real-time machine vision

Author

Listed:
  • Boyuan Cui

    (South China Normal University)

  • Zhen Fan

    (South China Normal University)

  • Wenjie Li

    (South China Normal University)

  • Yihong Chen

    (South China Normal University)

  • Shuai Dong

    (South China Normal University)

  • Zhengwei Tan

    (South China Normal University)

  • Shengliang Cheng

    (South China Normal University)

  • Bobo Tian

    (East China Normal University)

  • Ruiqiang Tao

    (South China Normal University)

  • Guo Tian

    (South China Normal University)

  • Deyang Chen

    (South China Normal University)

  • Zhipeng Hou

    (South China Normal University)

  • Minghui Qin

    (South China Normal University)

  • Min Zeng

    (South China Normal University)

  • Xubing Lu

    (South China Normal University)

  • Guofu Zhou

    (South China Normal University)

  • Xingsen Gao

    (South China Normal University)

  • Jun-Ming Liu

    (South China Normal University
    Nanjing University)

Abstract

Nowadays the development of machine vision is oriented toward real-time applications such as autonomous driving. This demands a hardware solution with low latency, high energy efficiency, and good reliability. Here, we demonstrate a robust and self-powered in-sensor computing paradigm with a ferroelectric photosensor network (FE-PS-NET). The FE-PS-NET, constituted by ferroelectric photosensors (FE-PSs) with tunable photoresponsivities, is capable of simultaneously capturing and processing images. In each FE-PS, self-powered photovoltaic responses, modulated by remanent polarization of an epitaxial ferroelectric Pb(Zr0.2Ti0.8)O3 layer, show not only multiple nonvolatile levels but also sign reversibility, enabling the representation of a signed weight in a single device and hence reducing the hardware overhead for network construction. With multiple FE-PSs wired together, the FE-PS-NET acts on its own as an artificial neural network. In situ multiply-accumulate operation between an input image and a stored photoresponsivity matrix is demonstrated in the FE-PS-NET. Moreover, the FE-PS-NET is faultlessly competent for real-time image processing functionalities, including binary classification between ‘X’ and ‘T’ patterns with 100% accuracy and edge detection for an arrow sign with an F-Measure of 1 (under 365 nm ultraviolet light). This study highlights the great potential of ferroelectric photovoltaics as the hardware basis of real-time machine vision.

Suggested Citation

  • Boyuan Cui & Zhen Fan & Wenjie Li & Yihong Chen & Shuai Dong & Zhengwei Tan & Shengliang Cheng & Bobo Tian & Ruiqiang Tao & Guo Tian & Deyang Chen & Zhipeng Hou & Minghui Qin & Min Zeng & Xubing Lu & , 2022. "Ferroelectric photosensor network: an advanced hardware solution to real-time machine vision," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
  • Handle: RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-022-29364-8
    DOI: 10.1038/s41467-022-29364-8
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-022-29364-8
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-022-29364-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Lukas Mennel & Joanna Symonowicz & Stefan Wachter & Dmitry K. Polyushkin & Aday J. Molina-Mendoza & Thomas Mueller, 2020. "Ultrafast machine vision with 2D material neural network image sensors," Nature, Nature, vol. 579(7797), pages 62-66, March.
    2. Fabien Alibart & Elham Zamanidoost & Dmitri B. Strukov, 2013. "Pattern classification by memristive crossbar circuits using ex situ and in situ training," Nature Communications, Nature, vol. 4(1), pages 1-7, October.
    3. Yang Chai, 2020. "In-sensor computing for machine vision," Nature, Nature, vol. 579(7797), pages 32-33, March.
    4. Chao Ma & Zhen Luo & Weichuan Huang & Letian Zhao & Qiaoling Chen & Yue Lin & Xiang Liu & Zhiwei Chen & Chuanchuan Liu & Haoyang Sun & Xi Jin & Yuewei Yin & Xiaoguang Li, 2020. "Sub-nanosecond memristor based on ferroelectric tunnel junction," Nature Communications, Nature, vol. 11(1), pages 1-9, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rengjian Yu & Lihua He & Changsong Gao & Xianghong Zhang & Enlong Li & Tailiang Guo & Wenwu Li & Huipeng Chen, 2022. "Programmable ferroelectric bionic vision hardware with selective attention for high-precision image classification," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    2. Changsong Gao & Di Liu & Chenhui Xu & Weidong Xie & Xianghong Zhang & Junhua Bai & Zhixian Lin & Cheng Zhang & Yuanyuan Hu & Tailiang Guo & Huipeng Chen, 2024. "Toward grouped-reservoir computing: organic neuromorphic vertical transistor with distributed reservoir states for efficient recognition and prediction," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    3. Zhiwei Chen & Wenjie Li & Zhen Fan & Shuai Dong & Yihong Chen & Minghui Qin & Min Zeng & Xubing Lu & Guofu Zhou & Xingsen Gao & Jun-Ming Liu, 2023. "All-ferroelectric implementation of reservoir computing," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    4. Haipeng Lin & Jiali Ou & Zhen Fan & Xiaobing Yan & Wenjie Hu & Boyuan Cui & Jikang Xu & Wenjie Li & Zhiwei Chen & Biao Yang & Kun Liu & Linyuan Mo & Meixia Li & Xubing Lu & Guofu Zhou & Xingsen Gao & , 2025. "In situ training of an in-sensor artificial neural network based on ferroelectric photosensors," Nature Communications, Nature, vol. 16(1), pages 1-12, December.
    5. Tian Zhang & Xin Guo & Pan Wang & Xinyi Fan & Zichen Wang & Yan Tong & Decheng Wang & Limin Tong & Linjun Li, 2024. "High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network," Nature Communications, Nature, vol. 15(1), pages 1-10, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xun Han & Juan Tao & Yegang Liang & Feng Guo & Zhangsheng Xu & Wenqiang Wu & Jiahui Tong & Mengxiao Chen & Caofeng Pan & Jianhua Hao, 2024. "Ultraweak light-modulated heterostructure with bidirectional photoresponse for static and dynamic image perception," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    2. Ge Li & Donggang Xie & Qinghua Zhang & Mingzhen Zhang & Zhuohui Liu & Zheng Wang & Jiahui Xie & Erjia Guo & Meng He & Can Wang & Lin Gu & Guozhen Yang & Kuijuan Jin & Chen Ge, 2025. "Interface-engineered non-volatile visible-blind photodetector for in-sensor computing," Nature Communications, Nature, vol. 16(1), pages 1-10, December.
    3. Doeon Lee & Minseong Park & Yongmin Baek & Byungjoon Bae & Junseok Heo & Kyusang Lee, 2022. "In-sensor image memorization and encoding via optical neurons for bio-stimulus domain reduction toward visual cognitive processing," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    4. Tao Guo & Shasha Li & Y. Norman Zhou & Wei D. Lu & Yong Yan & Yimin A. Wu, 2024. "Interspecies-chimera machine vision with polarimetry for real-time navigation and anti-glare pattern recognition," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    5. Guangdong Zhou & Jie Li & Qunliang Song & Lidan Wang & Zhijun Ren & Bai Sun & Xiaofang Hu & Wenhua Wang & Gaobo Xu & Xiaodie Chen & Lan Cheng & Feichi Zhou & Shukai Duan, 2023. "Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    6. Seok Daniel Namgung & Ryeong Myeong Kim & Yae-Chan Lim & Jong Woo Lee & Nam Heon Cho & Hyeohn Kim & Jin-Suk Huh & Hanju Rhee & Sanghee Nah & Min-Kyu Song & Jang-Yeon Kwon & Ki Tae Nam, 2022. "Circularly polarized light-sensitive, hot electron transistor with chiral plasmonic nanoparticles," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    7. Yuyan Zhu & Yang Wang & Xingchen Pang & Yongbo Jiang & Xiaoxian Liu & Qing Li & Zhen Wang & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile 2D MoS2/black phosphorus heterojunction photodiodes in the near- to mid-infrared region," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    8. Zhixin Yao & Huifeng Tian & U. Sasaki & Huacong Sun & Jingyi Hu & Guodong Xue & Ye Seul Jung & Ruijie Li & Zhenjiang Li & PeiChi Liao & Yihan Wang & Lina Yang Zhang & Ge Yin & Xuanyu Zhang & Yijie Luo, 2025. "Transferrable, wet-chemistry-derived high-k amorphous metal oxide dielectrics for two-dimensional electronic devices," Nature Communications, Nature, vol. 16(1), pages 1-11, December.
    9. Yao Ni & Jiaqi Liu & Hong Han & Qianbo Yu & Lu Yang & Zhipeng Xu & Chengpeng Jiang & Lu Liu & Wentao Xu, 2024. "Visualized in-sensor computing," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    10. Lei Tong & Yali Bi & Yilun Wang & Kai Peng & Xinyu Huang & Wei Ju & Zhuiri Peng & Zheng Li & Langlang Xu & Runfeng Lin & Xiangxiang Yu & Wenhao Shi & Hui Yu & Huajun Sun & Kanhao Xue & Qiang He & Ming, 2024. "Programmable nonlinear optical neuromorphic computing with bare 2D material MoS2," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    11. Jinlei Zhang & Jiayong Zhang & Yaping Qi & Shuainan Gong & Hang Xu & Zhenqi Liu & Ran Zhang & Mohammad A. Sadi & Demid Sychev & Run Zhao & Hongbin Yang & Zhenping Wu & Dapeng Cui & Lin Wang & Chunlan , 2024. "Room-temperature ferroelectric, piezoelectric and resistive switching behaviors of single-element Te nanowires," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    12. Alonso, F.J. & Maldonado, D. & Aguilera, A.M. & Roldán, J.B., 2021. "Memristor variability and stochastic physical properties modeling from a multivariate time series approach," Chaos, Solitons & Fractals, Elsevier, vol. 143(C).
    13. Maldonado, D. & Aguilera-Pedregosa, C. & Vinuesa, G. & García, H. & Dueñas, S. & Castán, H. & Aldana, S. & González, M.B. & Moreno, E. & Jiménez-Molinos, F. & Campabadal, F. & Roldán, J.B., 2022. "An experimental and simulation study of the role of thermal effects on variability in TiN/Ti/HfO2/W resistive switching nonlinear devices," Chaos, Solitons & Fractals, Elsevier, vol. 160(C).
    14. Xinyu Chen & Yufeng Xie & Yaochen Sheng & Hongwei Tang & Zeming Wang & Yu Wang & Yin Wang & Fuyou Liao & Jingyi Ma & Xiaojiao Guo & Ling Tong & Hanqi Liu & Hao Liu & Tianxiang Wu & Jiaxin Cao & Sitong, 2021. "Wafer-scale functional circuits based on two dimensional semiconductors with fabrication optimized by machine learning," Nature Communications, Nature, vol. 12(1), pages 1-8, December.
    15. Xingchen Pang & Yang Wang & Yuyan Zhu & Zhenhan Zhang & Du Xiang & Xun Ge & Haoqi Wu & Yongbo Jiang & Zizheng Liu & Xiaoxian Liu & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile rippled-assisted optoelectronic array for all-day motion detection and recognition," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    16. Bo Tong & Jiajun Xu & Jinhong Du & Peitao Liu & Tianming Du & Qiang Wang & Langjun Li & Yuning Wei & Jiangxu Li & Jinhua Liang & Chi Liu & Zhibo Liu & Chen Li & Lai-Peng Ma & Yang Chai & Wencai Ren, 2025. "2D (NH4)BiI3 enables non-volatile optoelectronic memories for machine learning," Nature Communications, Nature, vol. 16(1), pages 1-11, December.
    17. Haihui Lan & Luyang Wang & Runze He & Shuyi Huang & Jinqiu Yu & Jinming Guo & Jingrui Luo & Yiling Li & Jinyang Zhang & Jiaxin Lin & Shunping Zhang & Mengqi Zeng & Lei Fu, 2023. "2D quasi-layered material with domino structure," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    18. Yue Gong & Ruihuan Duan & Yi Hu & Yao Wu & Song Zhu & Xingli Wang & Qijie Wang & Shu Ping Lau & Zheng Liu & Beng Kang Tay, 2025. "Reconfigurable and nonvolatile ferroelectric bulk photovoltaics based on 3R-WS2 for machine vision," Nature Communications, Nature, vol. 16(1), pages 1-12, December.
    19. Tian Zhang & Xin Guo & Pan Wang & Xinyi Fan & Zichen Wang & Yan Tong & Decheng Wang & Limin Tong & Linjun Li, 2024. "High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    20. Hao Jiang & Yinzhu Chen & Wenyu Guo & Yan Zhang & Rigui Zhou & Mile Gu & Fan Zhong & Zhenhua Ni & Junpeng Lu & Cheng-Wei Qiu & Weibo Gao, 2024. "Metasurface-enabled broadband multidimensional photodetectors," Nature Communications, Nature, vol. 15(1), pages 1-10, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-022-29364-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.