IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v16y2025i1d10.1038_s41467-024-55558-3.html
   My bibliography  Save this article

Spectral convolutional neural network chip for in-sensor edge computing of incoherent natural light

Author

Listed:
  • Kaiyu Cui

    (Tsinghua University)

  • Shijie Rao

    (Tsinghua University)

  • Sheng Xu

    (Tsinghua University)

  • Yidong Huang

    (Tsinghua University)

  • Xusheng Cai

    (Beijing Seetrum Technology Co.)

  • Zhilei Huang

    (Beijing Seetrum Technology Co.)

  • Yu Wang

    (Beijing Seetrum Technology Co.)

  • Xue Feng

    (Tsinghua University)

  • Fang Liu

    (Tsinghua University)

  • Wei Zhang

    (Tsinghua University)

  • Yali Li

    (Tsinghua University)

  • Shengjin Wang

    (Tsinghua University)

Abstract

Optical neural networks are considered next-generation physical implementations of artificial neural networks, but their capabilities are limited by on-chip integration scale and requirement for coherent light sources. This study proposes a spectral convolutional neural network (SCNN) with matter meta-imaging. The optical convolutional layer is implemented by integrating very large-scale and pixel-aligned spectral filters on CMOS image sensor. It facilitates highly parallel spectral vector-inner products of incident incoherent natural light i.e., the direct information carrier, which empowers in-sensor optical analog computing at extremely high energy efficiency. To the best of our knowledge, this is the first integrated optical computing utilizing natural light. We employ the same SCNN chip for completely different real-world complex tasks and achieve accuracies of over 96% for pathological diagnosis and almost 100% for face anti-spoofing at video rates. These results indicate a feasible and scalable in-sensor edge computing chip of natural light for various portable terminals.

Suggested Citation

  • Kaiyu Cui & Shijie Rao & Sheng Xu & Yidong Huang & Xusheng Cai & Zhilei Huang & Yu Wang & Xue Feng & Fang Liu & Wei Zhang & Yali Li & Shengjin Wang, 2025. "Spectral convolutional neural network chip for in-sensor edge computing of incoherent natural light," Nature Communications, Nature, vol. 16(1), pages 1-10, December.
    • Handle: RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-024-55558-3
    DOI: 10.1038/s41467-024-55558-3
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-55558-3
    File Function: Abstract
    Download Restriction: no
    File URL: https://libkey.io/10.1038/s41467-024-55558-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Nicola Dietler & Matthias Minder & Vojislav Gligorovski & Augoustina Maria Economou & Denis Alain Henri Lucien Joly & Ahmad Sadeghi & Chun Hei Michael Chan & Mateusz Koziński & Martin Weigert & Anne-F, 2020. "A convolutional neural network segments yeast microscopy images with high accuracy," Nature Communications, Nature, vol. 11(1), pages 1-8, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lucas Henrion & Juan Andres Martinez & Vincent Vandenbroucke & Mathéo Delvenne & Samuel Telek & Andrew Zicler & Alexander Grünberger & Frank Delvigne, 2023. "Fitness cost associated with cell phenotypic switching drives population diversification dynamics and controllability," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    2. Kiyan Shabestary & Cinzia Klemm & Benedict Carling & James Marshall & Juline Savigny & Marko Storch & Rodrigo Ledesma-Amaro, 2024. "Phenotypic heterogeneity follows a growth-viability tradeoff in response to amino acid identity," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    3. Vojislav Gligorovski & Ahmad Sadeghi & Sahand Jamal Rahi, 2023. "Multidimensional characterization of inducible promoters and a highly light-sensitive LOV-transcription factor," Nature Communications, Nature, vol. 14(1), pages 1-18, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-024-55558-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.