IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-48099-2.html
   My bibliography  Save this article

Integrated photonic encoder for low power and high-speed image processing

Author

Listed:
  • Xiao Wang

    (University of Arizona)

  • Brandon Redding

    (U.S. Naval Research Laboratory)

  • Nicholas Karl

    (Sandia National Laboratories)

  • Christopher Long

    (Sandia National Laboratories)

  • Zheyuan Zhu

    (University of Central Floria)

  • James Skowronek

    (University of Arizona)

  • Shuo Pang

    (University of Central Floria)

  • David Brady

    (University of Arizona)

  • Raktim Sarma

    (Sandia National Laboratories
    Sandia National Laboratories)

Abstract

Modern lens designs are capable of resolving greater than 10 gigapixels, while advances in camera frame-rate and hyperspectral imaging have made data acquisition rates of Terapixel/second a real possibility. The main bottlenecks preventing such high data-rate systems are power consumption and data storage. In this work, we show that analog photonic encoders could address this challenge, enabling high-speed image compression using orders-of-magnitude lower power than digital electronics. Our approach relies on a silicon-photonics front-end to compress raw image data, foregoing energy-intensive image conditioning and reducing data storage requirements. The compression scheme uses a passive disordered photonic structure to perform kernel-type random projections of the raw image data with minimal power consumption and low latency. A back-end neural network can then reconstruct the original images with structural similarity exceeding 90%. This scheme has the potential to process data streams exceeding Terapixel/second using less than 100 fJ/pixel, providing a path to ultra-high-resolution data and image acquisition systems.

Suggested Citation

  • Xiao Wang & Brandon Redding & Nicholas Karl & Christopher Long & Zheyuan Zhu & James Skowronek & Shuo Pang & David Brady & Raktim Sarma, 2024. "Integrated photonic encoder for low power and high-speed image processing," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-48099-2
    DOI: 10.1038/s41467-024-48099-2
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-48099-2
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-48099-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yitong Chen & Maimaiti Nazhamaiti & Han Xu & Yao Meng & Tiankuang Zhou & Guangpu Li & Jingtao Fan & Qi Wei & Jiamin Wu & Fei Qiao & Lu Fang & Qionghai Dai, 2023. "All-analog photoelectronic chip for high-speed vision tasks," Nature, Nature, vol. 623(7985), pages 48-57, November.
    2. Daryl T. Spencer & Tara Drake & Travis C. Briles & Jordan Stone & Laura C. Sinclair & Connor Fredrick & Qing Li & Daron Westly & B. Robert Ilic & Aaron Bluestone & Nicolas Volet & Tin Komljenovic & Li, 2018. "An optical-frequency synthesizer using integrated photonics," Nature, Nature, vol. 557(7703), pages 81-85, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mingming Nie & Kunpeng Jia & Yijun Xie & Shining Zhu & Zhenda Xie & Shu-Wei Huang, 2022. "Synthesized spatiotemporal mode-locking and photonic flywheel in multimode mesoresonators," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    2. Wenting Wang & Ping-Keng Lu & Abhinav Kumar Vinod & Deniz Turan & James F. McMillan & Hao Liu & Mingbin Yu & Dim-Lee Kwong & Mona Jarrahi & Chee Wei Wong, 2022. "Coherent terahertz radiation with 2.8-octave tunability through chip-scale photomixed microresonator optical parametric oscillation," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    3. Seyed Danial Hashemi & Sunil Mittal, 2024. "Floquet topological dissipative Kerr solitons and incommensurate frequency combs," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    4. Dong-Chel Shin & Byung Soo Kim & Heesuk Jang & Young-Jin Kim & Seung-Woo Kim, 2023. "Photonic comb-rooted synthesis of ultra-stable terahertz frequencies," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    5. Chengying Bao & Zhiquan Yuan & Lue Wu & Myoung-Gyun Suh & Heming Wang & Qiang Lin & Kerry J. Vahala, 2021. "Architecture for microcomb-based GHz-mid-infrared dual-comb spectroscopy," Nature Communications, Nature, vol. 12(1), pages 1-8, December.
    6. Baheej Bathish & Raanan Gad & Fan Cheng & Kristoffer Karlsson & Ramgopal Madugani & Mark Douvidzon & Síle Nic Chormaic & Tal Carmon, 2023. "Absorption-induced transmission in plasma microphotonics," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    7. Xinxin Gao & Ze Gu & Qian Ma & Bao Jie Chen & Kam-Man Shum & Wen Yi Cui & Jian Wei You & Tie Jun Cui & Chi Hou Chan, 2024. "Terahertz spoof plasmonic neural network for diffractive information recognition and processing," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    8. Robert M. Gray & Mingchen Liu & Selina Zhou & Arkadev Roy & Luis Ledezma & Alireza Marandi, 2024. "Quadratic-soliton-enhanced mid-IR molecular sensing," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    9. Chao Xiang & Joel Guo & Warren Jin & Lue Wu & Jonathan Peters & Weiqiang Xie & Lin Chang & Boqiang Shen & Heming Wang & Qi-Fan Yang & David Kinghorn & Mario Paniccia & Kerry J. Vahala & Paul A. Morton, 2021. "High-performance lasers for fully integrated silicon nitride photonics," Nature Communications, Nature, vol. 12(1), pages 1-8, December.
    10. Xuguang Zhang & Zixuan Zhou & Yijun Guo & Minxue Zhuang & Warren Jin & Bitao Shen & Yujun Chen & Jiahui Huang & Zihan Tao & Ming Jin & Ruixuan Chen & Zhangfeng Ge & Zhou Fang & Ning Zhang & Yadong Liu, 2024. "High-coherence parallelization in integrated photonics," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    11. Jingwei Ling & Zhengdong Gao & Shixin Xue & Qili Hu & Mingxiao Li & Kaibo Zhang & Usman A. Javid & Raymond Lopez-Rios & Jeremy Staffa & Qiang Lin, 2024. "Electrically empowered microcomb laser," Nature Communications, Nature, vol. 15(1), pages 1-8, December.
    12. Miles H. Anderson & Wenle Weng & Grigory Lihachev & Alexey Tikan & Junqiu Liu & Tobias J. Kippenberg, 2022. "Zero dispersion Kerr solitons in optical microresonators," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    13. Alexandre Heintz & Bouraoui Ilahi & Alexandre Pofelski & Gianluigi Botton & Gilles Patriarche & Andrea Barzaghi & Simon Fafard & Richard Arès & Giovanni Isella & Abderraouf Boucherif, 2022. "Defect free strain relaxation of microcrystals on mesoporous patterned silicon," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    14. Su-Peng Yu & Erwan Lucas & Jizhao Zang & Scott B. Papp, 2022. "A continuum of bright and dark-pulse states in a photonic-crystal resonator," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    15. Chenghao Lao & Xing Jin & Lin Chang & Heming Wang & Zhe Lv & Weiqiang Xie & Haowen Shu & Xingjun Wang & John E. Bowers & Qi-Fan Yang, 2023. "Quantum decoherence of dark pulses in optical microresonators," Nature Communications, Nature, vol. 14(1), pages 1-8, December.
    16. Grigory Lihachev & Wenle Weng & Junqiu Liu & Lin Chang & Joel Guo & Jijun He & Rui Ning Wang & Miles H. Anderson & Yang Liu & John E. Bowers & Tobias J. Kippenberg, 2022. "Platicon microcomb generation using laser self-injection locking," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    17. Dmitry Kazakov & Theodore P. Letsou & Maximilian Beiser & Yiyang Zhi & Nikola Opačak & Marco Piccardo & Benedikt Schwarz & Federico Capasso, 2024. "Active mid-infrared ring resonators," Nature Communications, Nature, vol. 15(1), pages 1-8, December.
    18. Gregory Moille & Edgar F. Perez & Jordan R. Stone & Ashutosh Rao & Xiyuan Lu & Tahmid Sami Rahman & Yanne K. Chembo & Kartik Srinivasan, 2021. "Ultra-broadband Kerr microcomb through soliton spectral translation," Nature Communications, Nature, vol. 12(1), pages 1-9, December.
    19. Yinan Zhang & Shengting Zhu & Jinming Hu & Min Gu, 2024. "Femtosecond laser direct nanolithography of perovskite hydration for temporally programmable holograms," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    20. Zhenjia Chen & Zhenyuan Lin & Ji Yang & Cong Chen & Di Liu & Liuting Shan & Yuanyuan Hu & Tailiang Guo & Huipeng Chen, 2024. "Cross-layer transmission realized by light-emitting memristor for constructing ultra-deep neural network with transfer learning ability," Nature Communications, Nature, vol. 15(1), pages 1-12, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-48099-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.