IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-50271-7.html
   My bibliography  Save this article

Anti-distortion bioinspired camera with an inhomogeneous photo-pixel array

Author

Listed:
  • Changsoon Choi

    (Harvard University
    Korea Institute of Science and Technology (KIST))

  • Henry Hinton

    (Harvard University)

  • Hyojin Seung

    (Institute for Basic Science (IBS)
    Seoul National University)

  • Sehui Chang

    (Gwangju Institute of Science and Technology)

  • Ji Su Kim

    (Institute for Basic Science (IBS)
    Seoul National University)

  • Woosang You

    (Institute for Basic Science (IBS)
    Seoul National University)

  • Min Sung Kim

    (Institute for Basic Science (IBS)
    Seoul National University)

  • Jung Pyo Hong

    (Korea Institute of Science and Technology (KIST)
    Korea University)

  • Jung Ah Lim

    (Korea Institute of Science and Technology (KIST)
    University of Science and Technology (UST))

  • Do Kyung Hwang

    (Korea Institute of Science and Technology (KIST)
    Korea University)

  • Gil Ju Lee

    (Gwangju Institute of Science and Technology
    Pusan National University)

  • Houk Jang

    (Harvard University
    Brookhaven National Laboratory)

  • Young Min Song

    (Gwangju Institute of Science and Technology)

  • Dae-Hyeong Kim

    (Institute for Basic Science (IBS)
    Seoul National University)

  • Donhee Ham

    (Harvard University)

Abstract

The bioinspired camera, comprising a single lens and a curved image sensor—a photodiode array on a curved surface—, was born of flexible electronics. Its economical build lends itself well to space-constrained machine vision applications. The curved sensor, much akin to the retina, helps image focusing, but the curvature also creates a problem of image distortion, which can undermine machine vision tasks such as object recognition. Here we report an anti-distortion single-lens camera, where 4096 silicon photodiodes arrayed on a curved surface in a nonuniform pattern assimilated to the distorting optics are the key to anti-distortion engineering. That is, the photo-pixel distribution pattern itself is warped in the same manner as images are warped, which correctively reverses distortion. Acquired images feature no appreciable distortion across a 120° horizontal view, as confirmed by their neural-network recognition accuracies. This distortion correction via photo-pixel array reconfiguration is a form of in-sensor computing.

Suggested Citation

  • Changsoon Choi & Henry Hinton & Hyojin Seung & Sehui Chang & Ji Su Kim & Woosang You & Min Sung Kim & Jung Pyo Hong & Jung Ah Lim & Do Kyung Hwang & Gil Ju Lee & Houk Jang & Young Min Song & Dae-Hyeon, 2024. "Anti-distortion bioinspired camera with an inhomogeneous photo-pixel array," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-50271-7
    DOI: 10.1038/s41467-024-50271-7
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-50271-7
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-50271-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Heung Cho Ko & Mark P. Stoykovich & Jizhou Song & Viktor Malyarchuk & Won Mook Choi & Chang-Jae Yu & Joseph B. Geddes III & Jianliang Xiao & Shuodao Wang & Yonggang Huang & John A. Rogers, 2008. "A hemispherical electronic eye camera based on compressible silicon optoelectronics," Nature, Nature, vol. 454(7205), pages 748-753, August.
    2. Wonho Lee & Yuan Liu & Yongjun Lee & Bhupendra K. Sharma & Sachin M. Shinde & Seong Dae Kim & Kewang Nan & Zheng Yan & Mengdi Han & Yonggang Huang & Yihui Zhang & Jong-Hyun Ahn & John A. Rogers, 2018. "Two-dimensional materials in functional three-dimensional architectures with applications in photodetection and imaging," Nature Communications, Nature, vol. 9(1), pages 1-9, December.
    3. Changsoon Choi & Moon Kee Choi & Siyi Liu & Minsung Kim & Ok Kyu Park & Changkyun Im & Jaemin Kim & Xiaoliang Qin & Gil Ju Lee & Kyoung Won Cho & Myungbin Kim & Eehyung Joh & Jongha Lee & Donghee Son , 2017. "Human eye-inspired soft optoelectronic device using high-density MoS2-graphene curved image sensor array," Nature Communications, Nature, vol. 8(1), pages 1-11, December.
    4. Doeon Lee & Minseong Park & Yongmin Baek & Byungjoon Bae & Junseok Heo & Kyusang Lee, 2022. "In-sensor image memorization and encoding via optical neurons for bio-stimulus domain reduction toward visual cognitive processing," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    5. Kan Zhang & Yei Hwan Jung & Solomon Mikael & Jung-Hun Seo & Munho Kim & Hongyi Mi & Han Zhou & Zhenyang Xia & Weidong Zhou & Shaoqin Gong & Zhenqiang Ma, 2017. "Origami silicon optoelectronics for hemispherical electronic eye systems," Nature Communications, Nature, vol. 8(1), pages 1-8, December.
    6. Lukas Mennel & Joanna Symonowicz & Stefan Wachter & Dmitry K. Polyushkin & Aday J. Molina-Mendoza & Thomas Mueller, 2020. "Ultrafast machine vision with 2D material neural network image sensors," Nature, Nature, vol. 579(7797), pages 62-66, March.
    7. Leilei Gu & Swapnadeep Poddar & Yuanjing Lin & Zhenghao Long & Daquan Zhang & Qianpeng Zhang & Lei Shu & Xiao Qiu & Matthew Kam & Ali Javey & Zhiyong Fan, 2020. "A biomimetic eye with a hemispherical perovskite nanowire array retina," Nature, Nature, vol. 581(7808), pages 278-282, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhenghao Long & Xiao Qiu & Chak Lam Jonathan Chan & Zhibo Sun & Zhengnan Yuan & Swapnadeep Poddar & Yuting Zhang & Yucheng Ding & Leilei Gu & Yu Zhou & Wenying Tang & Abhishek Kumar Srivastava & Cunji, 2023. "A neuromorphic bionic eye with filter-free color vision using hemispherical perovskite nanowire array retina," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    2. Bo Dai & Liang Zhang & Chenglong Zhao & Hunter Bachman & Ryan Becker & John Mai & Ziao Jiao & Wei Li & Lulu Zheng & Xinjun Wan & Tony Jun Huang & Songlin Zhuang & Dawei Zhang, 2021. "Biomimetic apposition compound eye fabricated using microfluidic-assisted 3D printing," Nature Communications, Nature, vol. 12(1), pages 1-11, December.
    3. Ming Deng & Ziqing Li & Shiyuan Liu & Xiaosheng Fang & Limin Wu, 2024. "Wafer-scale integration of two-dimensional perovskite oxides towards motion recognition," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    4. Xu Luo & Chen Chen & Zixi He & Min Wang & Keyuan Pan & Xuemei Dong & Zifan Li & Bin Liu & Zicheng Zhang & Yueyue Wu & Chaoyi Ban & Rong Chen & Dengfeng Zhang & Kaili Wang & Qiye Wang & Junyue Li & Gan, 2024. "A bionic self-driven retinomorphic eye with ionogel photosynaptic retina," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    5. Xiaopeng Feng & Chenglong Li & Jinmei Song & Yuhong He & Wei Qu & Weijun Li & Keke Guo & Lulu Liu & Bai Yang & Haotong Wei, 2024. "Differential perovskite hemispherical photodetector for intelligent imaging and location tracking," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    6. Guangdong Zhou & Jie Li & Qunliang Song & Lidan Wang & Zhijun Ren & Bai Sun & Xiaofang Hu & Wenhua Wang & Gaobo Xu & Xiaodie Chen & Lan Cheng & Feichi Zhou & Shukai Duan, 2023. "Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    7. Zhuiri Peng & Lei Tong & Wenhao Shi & Langlang Xu & Xinyu Huang & Zheng Li & Xiangxiang Yu & Xiaohan Meng & Xiao He & Shengjie Lv & Gaochen Yang & Hao Hao & Tian Jiang & Xiangshui Miao & Lei Ye, 2024. "Multifunctional human visual pathway-replicated hardware based on 2D materials," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    8. Tian Zhang & Xin Guo & Pan Wang & Xinyi Fan & Zichen Wang & Yan Tong & Decheng Wang & Limin Tong & Linjun Li, 2024. "High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    9. Yuyan Zhu & Yang Wang & Xingchen Pang & Yongbo Jiang & Xiaoxian Liu & Qing Li & Zhen Wang & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile 2D MoS2/black phosphorus heterojunction photodiodes in the near- to mid-infrared region," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    10. Tao Guo & Shasha Li & Y. Norman Zhou & Wei D. Lu & Yong Yan & Yimin A. Wu, 2024. "Interspecies-chimera machine vision with polarimetry for real-time navigation and anti-glare pattern recognition," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    11. Cao, Dong-Xing & Lu, Yi-Ming & Lai, Siu-Kai & Mao, Jia-Jia & Guo, Xiang-Ying & Shen, Yong-Jun, 2022. "A novel soft encapsulated multi-directional and multi-modal piezoelectric vibration energy harvester," Energy, Elsevier, vol. 254(PB).
    12. Yan Sun & Shuting Xu & Zheqi Xu & Jiamin Tian & Mengmeng Bai & Zhiying Qi & Yue Niu & Hein Htet Aung & Xiaolu Xiong & Junfeng Han & Cuicui Lu & Jianbo Yin & Sheng Wang & Qing Chen & Reshef Tenne & All, 2022. "Mesoscopic sliding ferroelectricity enabled photovoltaic random access memory for material-level artificial vision system," Nature Communications, Nature, vol. 13(1), pages 1-8, December.
    13. Xiaopeng Feng & Yuhong He & Wei Qu & Jinmei Song & Wanting Pan & Mingrui Tan & Bai Yang & Haotong Wei, 2022. "Spray-coated perovskite hemispherical photodetector featuring narrow-band and wide-angle imaging," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    14. Rui Xu & Zhiqiang Zeng & Yong Lei, 2022. "Well-defined nanostructuring with designable anodic aluminum oxide template," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    15. Pengshan Xie & Yunchao Xu & Jingwen Wang & Dengji Li & Yuxuan Zhang & Zixin Zeng & Boxiang Gao & Quan Quan & Bowen Li & You Meng & Weijun Wang & Yezhan Li & Yan Yan & Yi Shen & Jia Sun & Johnny C. Ho, 2024. "Birdlike broadband neuromorphic visual sensor arrays for fusion imaging," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    16. Pei-Yu Huang & Bi-Yi Jiang & Hong-Ji Chen & Jia-Yi Xu & Kang Wang & Cheng-Yi Zhu & Xin-Yan Hu & Dong Li & Liang Zhen & Fei-Chi Zhou & Jing-Kai Qin & Cheng-Yan Xu, 2023. "Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    17. You Meng & Xiaocui Li & Xiaolin Kang & Wanpeng Li & Wei Wang & Zhengxun Lai & Weijun Wang & Quan Quan & Xiuming Bu & SenPo Yip & Pengshan Xie & Dong Chen & Dengji Li & Fei Wang & Chi-Fung Yeung & Chan, 2023. "Van der Waals nanomesh electronics on arbitrary surfaces," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    18. Junxiong Guo & Shuyi Gu & Lin Lin & Yu Liu & Ji Cai & Hongyi Cai & Yu Tian & Yuelin Zhang & Qinghua Zhang & Ze Liu & Yafei Zhang & Xiaosheng Zhang & Yuan Lin & Wen Huang & Lin Gu & Jinxing Zhang, 2024. "Type-printable photodetector arrays for multichannel meta-infrared imaging," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    19. Ling Li & Shasha Li & Wenhai Wang & Jielian Zhang & Yiming Sun & Qunrui Deng & Tao Zheng & Jianting Lu & Wei Gao & Mengmeng Yang & Hanyu Wang & Yuan Pan & Xueting Liu & Yani Yang & Jingbo Li & Nengjie, 2024. "Adaptative machine vision with microsecond-level accurate perception beyond human retina," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    20. Xinxin Gao & Ze Gu & Qian Ma & Bao Jie Chen & Kam-Man Shum & Wen Yi Cui & Jian Wei You & Tie Jun Cui & Chi Hou Chan, 2024. "Terahertz spoof plasmonic neural network for diffractive information recognition and processing," Nature Communications, Nature, vol. 15(1), pages 1-11, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-50271-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.