IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v13y2022i1d10.1038_s41467-022-29568-y.html
   My bibliography  Save this article

Trilobite-inspired neural nanophotonic light-field camera with extreme depth-of-field

Author

Listed:
  • Qingbin Fan

    (Nanjing University
    Nanjing University
    Nanjing University)

  • Weizhu Xu

    (Nanjing University
    Nanjing University)

  • Xuemei Hu

    (Nanjing University
    Nanjing University)

  • Wenqi Zhu

    (National Institute of Standards and Technology
    University of Maryland)

  • Tao Yue

    (Nanjing University
    Nanjing University)

  • Cheng Zhang

    (Huazhong University of Science and Technology)

  • Feng Yan

    (Nanjing University
    Nanjing University)

  • Lu Chen

    (National Institute of Standards and Technology
    University of Maryland)

  • Henri J. Lezec

    (National Institute of Standards and Technology)

  • Yanqing Lu

    (Nanjing University
    Nanjing University)

  • Amit Agrawal

    (National Institute of Standards and Technology
    University of Maryland)

  • Ting Xu

    (Nanjing University
    Nanjing University)

Abstract

A unique bifocal compound eye visual system found in the now extinct trilobite, Dalmanitina socialis, may enable them to be sensitive to the light-field information and simultaneously perceive both close and distant objects in the environment. Here, inspired by the optical structure of their eyes, we demonstrate a nanophotonic light-field camera incorporating a spin-multiplexed bifocal metalens array capable of capturing high-resolution light-field images over a record depth-of-field ranging from centimeter to kilometer scale, simultaneously enabling macro and telephoto modes in a snapshot imaging. By leveraging a multi-scale convolutional neural network-based reconstruction algorithm, optical aberrations induced by the metalens are eliminated, thereby significantly relaxing the design and performance limitations on metasurface optics. The elegant integration of nanophotonic technology with computational photography achieved here is expected to aid development of future high-performance imaging systems.

Suggested Citation

  • Qingbin Fan & Weizhu Xu & Xuemei Hu & Wenqi Zhu & Tao Yue & Cheng Zhang & Feng Yan & Lu Chen & Henri J. Lezec & Yanqing Lu & Amit Agrawal & Ting Xu, 2022. "Trilobite-inspired neural nanophotonic light-field camera with extreme depth-of-field," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
  • Handle: RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-022-29568-y
    DOI: 10.1038/s41467-022-29568-y
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-022-29568-y
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-022-29568-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Zhenyu Yang & Zhaokun Wang & Yuxi Wang & Xing Feng & Ming Zhao & Zhujun Wan & Liangqiu Zhu & Jun Liu & Yi Huang & Jinsong Xia & Martin Wegener, 2018. "Generalized Hartmann-Shack array of dielectric metalens sub-arrays for polarimetric beam profiling," Nature Communications, Nature, vol. 9(1), pages 1-7, December.
    2. Leonard C. Kogos & Yunzhe Li & Jianing Liu & Yuyu Li & Lei Tian & Roberto Paiella, 2020. "Plasmonic ommatidia for lensless compound-eye vision," Nature Communications, Nature, vol. 11(1), pages 1-9, December.
    3. Xingjie Ni & Alexander V. Kildishev & Vladimir M. Shalaev, 2013. "Metasurface holograms for visible light," Nature Communications, Nature, vol. 4(1), pages 1-6, December.
    4. Amir Arbabi & Ehsan Arbabi & Seyedeh Mahsa Kamali & Yu Horie & Seunghoon Han & Andrei Faraon, 2016. "Miniature optical planar camera based on a wide-angle metasurface doublet corrected for monochromatic aberrations," Nature Communications, Nature, vol. 7(1), pages 1-9, December.
    5. Ethan Tseng & Shane Colburn & James Whitehead & Luocheng Huang & Seung-Hwan Baek & Arka Majumdar & Felix Heide, 2021. "Neural nano-optics for high-quality thin lens imaging," Nature Communications, Nature, vol. 12(1), pages 1-7, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiaopeng Feng & Yuhong He & Wei Qu & Jinmei Song & Wanting Pan & Mingrui Tan & Bai Yang & Haotong Wei, 2022. "Spray-coated perovskite hemispherical photodetector featuring narrow-band and wide-angle imaging," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    2. Yueqiang Hu & Yuting Jiang & Yi Zhang & Xing Yang & Xiangnian Ou & Ling Li & Xianghong Kong & Xingsi Liu & Cheng-Wei Qiu & Huigao Duan, 2023. "Asymptotic dispersion engineering for ultra-broadband meta-optics," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    3. Zhaoyi Li & Raphaël Pestourie & Joon-Suh Park & Yao-Wei Huang & Steven G. Johnson & Federico Capasso, 2022. "Inverse design enables large-scale high-performance meta-optics reshaping virtual reality," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    4. Brandon Born & Sung-Hoon Lee & Jung-Hwan Song & Jeong Yub Lee & Woong Ko & Mark L. Brongersma, 2023. "Off-axis metasurfaces for folded flat optics," Nature Communications, Nature, vol. 14(1), pages 1-8, December.
    5. Xia Hua & Yujie Wang & Shuming Wang & Xiujuan Zou & You Zhou & Lin Li & Feng Yan & Xun Cao & Shumin Xiao & Din Ping Tsai & Jiecai Han & Zhenlin Wang & Shining Zhu, 2022. "Ultra-compact snapshot spectral light-field imaging," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    6. Yue Wu & Ji Chen & Yin Wang & Zhongyi Yuan & Chunyu Huang & Jiacheng Sun & Chengyi Feng & Muyang Li & Kai Qiu & Shining Zhu & Zaichen Zhang & Tao Li, 2024. "Tbps wide-field parallel optical wireless communications based on a metasurface beam splitter," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    7. Xiujuan Zou & Youming Zhang & Ruoyu Lin & Guangxing Gong & Shuming Wang & Shining Zhu & Zhenlin Wang, 2022. "Pixel-level Bayer-type colour router based on metasurfaces," Nature Communications, Nature, vol. 13(1), pages 1-7, December.
    8. Xinge Yang & Qiang Fu & Wolfgang Heidrich, 2024. "Curriculum learning for ab initio deep learned refractive optics," Nature Communications, Nature, vol. 15(1), pages 1-8, December.
    9. Gang Wu & Mohamed Abid & Mohamed Zerara & Jiung Cho & Miri Choi & Cormac Ó Coileáin & Kuan-Ming Hung & Ching-Ray Chang & Igor V. Shvets & Han-Chun Wu, 2024. "Miniaturized spectrometer with intrinsic long-term image memory," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    10. Tianshuo Qiu & Qiang An & Jianqi Wang & Jiafu Wang & Cheng-Wei Qiu & Shiyong Li & Hao Lv & Ming Cai & Jianyi Wang & Lin Cong & Shaobo Qu, 2024. "Vision-driven metasurfaces for perception enhancement," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    11. Zicheng Shen & Feng Zhao & Chunqi Jin & Shuai Wang & Liangcai Cao & Yuanmu Yang, 2023. "Monocular metasurface camera for passive single-shot 4D imaging," Nature Communications, Nature, vol. 14(1), pages 1-8, December.
    12. Junhui Hu & Zeyuan Guo & Jianyang Shi & Xiong Jiang & Qinmiao Chen & Hui Chen & Zhixue He & Qinghai Song & Shumin Xiao & Shaohua Yu & Nan Chi & Chao Shen, 2024. "A metasurface-based full-color circular auto-focusing Airy beam transmitter for stable high-speed underwater wireless optical communications," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    13. Zi Wang & Lorry Chang & Feifan Wang & Tiantian Li & Tingyi Gu, 2022. "Integrated photonic metasystem for image classifications at telecommunication wavelength," Nature Communications, Nature, vol. 13(1), pages 1-8, December.
    14. Corey A. Richards & Christian R. Ocier & Dajie Xie & Haibo Gao & Taylor Robertson & Lynford L. Goddard & Rasmus E. Christiansen & David G. Cahill & Paul V. Braun, 2023. "Hybrid achromatic microlenses with high numerical apertures and focusing efficiencies across the visible," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    15. Claudio U. Hail & Morgan Foley & Ruzan Sokhoyan & Lior Michaeli & Harry A. Atwater, 2023. "High quality factor metasurfaces for two-dimensional wavefront manipulation," Nature Communications, Nature, vol. 14(1), pages 1-8, December.
    16. Zong-Lin Li & Kun Chen & Fei Li & Zhi-Jun Shi & Qi-Li Sun & Peng-Qi Li & Yu-Gui Peng & Lai-Xin Huang & Guang Yang & Hairong Zheng & Xue-Feng Zhu, 2023. "Decorated bacteria-cellulose ultrasonic metasurface," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    17. Yuanlong Zhang & Xiaofei Song & Jiachen Xie & Jing Hu & Jiawei Chen & Xiang Li & Haiyu Zhang & Qiqun Zhou & Lekang Yuan & Chui Kong & Yibing Shen & Jiamin Wu & Lu Fang & Qionghai Dai, 2023. "Large depth-of-field ultra-compact microscope by progressive optimization and deep learning," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    18. Georgy Ermolaev & Kirill Voronin & Denis G. Baranov & Vasyl Kravets & Gleb Tselikov & Yury Stebunov & Dmitry Yakubovsky & Sergey Novikov & Andrey Vyshnevyy & Arslan Mazitov & Ivan Kruglov & Sergey Zhu, 2022. "Topological phase singularities in atomically thin high-refractive-index materials," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    19. Ethan Tseng & Shane Colburn & James Whitehead & Luocheng Huang & Seung-Hwan Baek & Arka Majumdar & Felix Heide, 2021. "Neural nano-optics for high-quality thin lens imaging," Nature Communications, Nature, vol. 12(1), pages 1-7, December.
    20. Weihan Li & Qian Ma & Che Liu & Yunfeng Zhang & Xianning Wu & Jiawei Wang & Shizhao Gao & Tianshuo Qiu & Tonghao Liu & Qiang Xiao & Jiaxuan Wei & Ting Ting Gu & Zhize Zhou & Fashuai Li & Qiang Cheng &, 2023. "Intelligent metasurface system for automatic tracking of moving targets and wireless communications based on computer vision," Nature Communications, Nature, vol. 14(1), pages 1-10, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-022-29568-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.