IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v12y2021i1d10.1038_s41467-021-26721-x.html
   My bibliography  Save this article

Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second

Author

Listed:
  • Ji Hyun Nam

    (University of Wisconsin – Madison)

  • Eric Brandt

    (University of Wisconsin – Madison)

  • Sebastian Bauer

    (University of Wisconsin – Madison)

  • Xiaochun Liu

    (University of Wisconsin – Madison)

  • Marco Renna

    (Politecnico di Milano)

  • Alberto Tosi

    (Politecnico di Milano)

  • Eftychios Sifakis

    (University of Wisconsin – Madison)

  • Andreas Velten

    (University of Wisconsin – Madison
    University of Wisconsin – Madison)

Abstract

Non-Line-Of-Sight (NLOS) imaging aims at recovering the 3D geometry of objects that are hidden from the direct line of sight. One major challenge with this technique is the weak available multibounce signal limiting scene size, capture speed, and reconstruction quality. To overcome this obstacle, we introduce a multipixel time-of-flight non-line-of-sight imaging method combining specifically designed Single Photon Avalanche Diode (SPAD) array detectors with a fast reconstruction algorithm that captures and reconstructs live low-latency videos of non-line-of-sight scenes with natural non-retroreflective objects. We develop a model of the signal-to-noise-ratio of non-line-of-sight imaging and use it to devise a method that reconstructs the scene such that signal-to-noise-ratio, motion blur, angular resolution, and depth resolution are all independent of scene depth suggesting that reconstruction of very large scenes may be possible.

Suggested Citation

  • Ji Hyun Nam & Eric Brandt & Sebastian Bauer & Xiaochun Liu & Marco Renna & Alberto Tosi & Eftychios Sifakis & Andreas Velten, 2021. "Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second," Nature Communications, Nature, vol. 12(1), pages 1-10, December.
    • Handle: RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-26721-x
    DOI: 10.1038/s41467-021-26721-x
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-021-26721-x
    File Function: Abstract
    Download Restriction: no
    File URL: https://libkey.io/10.1038/s41467-021-26721-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Charles Saunders & John Murray-Bruce & Vivek K Goyal, 2019. "Computational periscopy with an ordinary digital camera," Nature, Nature, vol. 565(7740), pages 472-475, January.
    2. Matthew O’Toole & David B. Lindell & Gordon Wetzstein, 2018. "Confocal non-line-of-sight imaging based on the light-cone transform," Nature, Nature, vol. 555(7696), pages 338-341, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robinson Czajkowski & John Murray-Bruce, 2024. "Two-edge-resolved three-dimensional non-line-of-sight imaging with an ordinary camera," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    2. Florian Willomitzer & Prasanna V. Rangarajan & Fengqiang Li & Muralidhar M. Balaji & Marc P. Christensen & Oliver Cossairt, 2021. "Fast non-line-of-sight imaging with high-resolution and wide field of view using synthetic wavelength holography," Nature Communications, Nature, vol. 12(1), pages 1-11, December.
    3. Xiaohua Feng & Yayao Ma & Liang Gao, 2022. "Compact light field photography towards versatile three-dimensional vision," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    4. Sheila Seidel & Hoover Rueda-Chacón & Iris Cusini & Federica Villa & Franco Zappa & Christopher Yu & Vivek K Goyal, 2023. "Non-line-of-sight snapshots and background mapping with an active corner camera," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    5. Tian Shi & Liangsheng Li & He Cai & Xianli Zhu & Qingfan Shi & Ning Zheng, 2022. "Computational imaging of moving objects obscured by a random corridor via speckle correlations," Nature Communications, Nature, vol. 13(1), pages 1-7, December.
    6. Xintong Liu & Jianyu Wang & Leping Xiao & Zuoqiang Shi & Xing Fu & Lingyun Qiu, 2023. "Non-line-of-sight imaging with arbitrary illumination and detection pattern," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    7. Sungsam Kang & Yongwoo Kwon & Hojun Lee & Seho Kim & Jin Hee Hong & Seokchan Yoon & Wonshik Choi, 2023. "Tracing multiple scattering trajectories for deep optical imaging in scattering media," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    8. Yaoyao Shi & Wei Sheng & Yangyang Fu & Youwen Liu, 2023. "Overlapping speckle correlation algorithm for high-resolution imaging and tracking of objects in unknown scattering media," Nature Communications, Nature, vol. 14(1), pages 1-8, December.
    9. Chung Il Park & Seungah Choe & Woorim Lee & Wonjae Choi & Miso Kim & Hong Min Seung & Yoon Young Kim, 2023. "Ultrasonic barrier-through imaging by Fabry-Perot resonance-tailoring panel," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    10. Md Sadman Sakib Rahman & Tianyi Gan & Emir Arda Deger & Çağatay Işıl & Mona Jarrahi & Aydogan Ozcan, 2023. "Learning diffractive optical communication around arbitrary opaque occlusions," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    11. Dongyu Du & Xin Jin & Rujia Deng & Jinshi Kang & Hongkun Cao & Yihui Fan & Zhiheng Li & Haoqian Wang & Xiangyang Ji & Jingyan Song, 2022. "A boundary migration model for imaging within volumetric scattering media," Nature Communications, Nature, vol. 13(1), pages 1-10, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-26721-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.