IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v12y2021i1d10.1038_s41467-021-26443-0.html
   My bibliography  Save this article

Neural nano-optics for high-quality thin lens imaging

Author

Listed:
  • Ethan Tseng

    (Department of Computer Science)

  • Shane Colburn

    (Department of Electrical and Computer Engineering)

  • James Whitehead

    (Department of Electrical and Computer Engineering)

  • Luocheng Huang

    (Department of Electrical and Computer Engineering)

  • Seung-Hwan Baek

    (Department of Computer Science)

  • Arka Majumdar

    (Department of Electrical and Computer Engineering
    Department of Physics)

  • Felix Heide

    (Department of Computer Science)

Abstract

Nano-optic imagers that modulate light at sub-wavelength scales could enable new applications in diverse domains ranging from robotics to medicine. Although metasurface optics offer a path to such ultra-small imagers, existing methods have achieved image quality far worse than bulky refractive alternatives, fundamentally limited by aberrations at large apertures and low f-numbers. In this work, we close this performance gap by introducing a neural nano-optics imager. We devise a fully differentiable learning framework that learns a metasurface physical structure in conjunction with a neural feature-based image reconstruction algorithm. Experimentally validating the proposed method, we achieve an order of magnitude lower reconstruction error than existing approaches. As such, we present a high-quality, nano-optic imager that combines the widest field-of-view for full-color metasurface operation while simultaneously achieving the largest demonstrated aperture of 0.5 mm at an f-number of 2.

Suggested Citation

  • Ethan Tseng & Shane Colburn & James Whitehead & Luocheng Huang & Seung-Hwan Baek & Arka Majumdar & Felix Heide, 2021. "Neural nano-optics for high-quality thin lens imaging," Nature Communications, Nature, vol. 12(1), pages 1-7, December.
  • Handle: RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-26443-0
    DOI: 10.1038/s41467-021-26443-0
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-021-26443-0
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-021-26443-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jacob Engelberg & Uriel Levy, 2020. "The advantages of metalenses over diffractive lenses," Nature Communications, Nature, vol. 11(1), pages 1-4, December.
    2. Ori Avayu & Euclides Almeida & Yehiam Prior & Tal Ellenbogen, 2017. "Composite functional metasurfaces for multispectral achromatic optics," Nature Communications, Nature, vol. 8(1), pages 1-7, April.
    3. Amir Arbabi & Ehsan Arbabi & Seyedeh Mahsa Kamali & Yu Horie & Seunghoon Han & Andrei Faraon, 2016. "Miniature optical planar camera based on a wide-angle metasurface doublet corrected for monochromatic aberrations," Nature Communications, Nature, vol. 7(1), pages 1-9, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xinge Yang & Qiang Fu & Wolfgang Heidrich, 2024. "Curriculum learning for ab initio deep learned refractive optics," Nature Communications, Nature, vol. 15(1), pages 1-8, December.
    2. Yuanlong Zhang & Xiaofei Song & Jiachen Xie & Jing Hu & Jiawei Chen & Xiang Li & Haiyu Zhang & Qiqun Zhou & Lekang Yuan & Chui Kong & Yibing Shen & Jiamin Wu & Lu Fang & Qionghai Dai, 2023. "Large depth-of-field ultra-compact microscope by progressive optimization and deep learning," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    3. Zhaoyi Li & Raphaël Pestourie & Joon-Suh Park & Yao-Wei Huang & Steven G. Johnson & Federico Capasso, 2022. "Inverse design enables large-scale high-performance meta-optics reshaping virtual reality," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    4. Gang Wu & Mohamed Abid & Mohamed Zerara & Jiung Cho & Miri Choi & Cormac Ó Coileáin & Kuan-Ming Hung & Ching-Ray Chang & Igor V. Shvets & Han-Chun Wu, 2024. "Miniaturized spectrometer with intrinsic long-term image memory," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    5. Qingbin Fan & Weizhu Xu & Xuemei Hu & Wenqi Zhu & Tao Yue & Cheng Zhang & Feng Yan & Lu Chen & Henri J. Lezec & Yanqing Lu & Amit Agrawal & Ting Xu, 2022. "Trilobite-inspired neural nanophotonic light-field camera with extreme depth-of-field," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    6. Corey A. Richards & Christian R. Ocier & Dajie Xie & Haibo Gao & Taylor Robertson & Lynford L. Goddard & Rasmus E. Christiansen & David G. Cahill & Paul V. Braun, 2023. "Hybrid achromatic microlenses with high numerical apertures and focusing efficiencies across the visible," Nature Communications, Nature, vol. 14(1), pages 1-11, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiaopeng Feng & Yuhong He & Wei Qu & Jinmei Song & Wanting Pan & Mingrui Tan & Bai Yang & Haotong Wei, 2022. "Spray-coated perovskite hemispherical photodetector featuring narrow-band and wide-angle imaging," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    2. Yueqiang Hu & Yuting Jiang & Yi Zhang & Xing Yang & Xiangnian Ou & Ling Li & Xianghong Kong & Xingsi Liu & Cheng-Wei Qiu & Huigao Duan, 2023. "Asymptotic dispersion engineering for ultra-broadband meta-optics," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    3. Brandon Born & Sung-Hoon Lee & Jung-Hwan Song & Jeong Yub Lee & Woong Ko & Mark L. Brongersma, 2023. "Off-axis metasurfaces for folded flat optics," Nature Communications, Nature, vol. 14(1), pages 1-8, December.
    4. Qingbin Fan & Weizhu Xu & Xuemei Hu & Wenqi Zhu & Tao Yue & Cheng Zhang & Feng Yan & Lu Chen & Henri J. Lezec & Yanqing Lu & Amit Agrawal & Ting Xu, 2022. "Trilobite-inspired neural nanophotonic light-field camera with extreme depth-of-field," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    5. Xia Hua & Yujie Wang & Shuming Wang & Xiujuan Zou & You Zhou & Lin Li & Feng Yan & Xun Cao & Shumin Xiao & Din Ping Tsai & Jiecai Han & Zhenlin Wang & Shining Zhu, 2022. "Ultra-compact snapshot spectral light-field imaging," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    6. Maryam Ghahremani & Andrew McClung & Babak Mirzapourbeinekalaye & Amir Arbabi, 2024. "3D alignment of distant patterns with deep-subwavelength precision using metasurfaces," Nature Communications, Nature, vol. 15(1), pages 1-8, December.
    7. Yue Wu & Ji Chen & Yin Wang & Zhongyi Yuan & Chunyu Huang & Jiacheng Sun & Chengyi Feng & Muyang Li & Kai Qiu & Shining Zhu & Zaichen Zhang & Tao Li, 2024. "Tbps wide-field parallel optical wireless communications based on a metasurface beam splitter," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    8. Zicheng Shen & Feng Zhao & Chunqi Jin & Shuai Wang & Liangcai Cao & Yuanmu Yang, 2023. "Monocular metasurface camera for passive single-shot 4D imaging," Nature Communications, Nature, vol. 14(1), pages 1-8, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-26443-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.