IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v15y2024i1d10.1038_s41467-024-53261-x.html
   My bibliography  Save this article

120 GOPS Photonic tensor core in thin-film lithium niobate for inference and in situ training

Author

Listed:
  • Zhongjin Lin

    (The University of British Columbia
    Sun Yat-sen University)

  • Bhavin J. Shastri

    (Queen’s University)

  • Shangxuan Yu

    (The University of British Columbia)

  • Jingxiang Song

    (The University of British Columbia)

  • Yuntao Zhu

    (Sun Yat-sen University)

  • Arman Safarnejadian

    (Université Laval)

  • Wangning Cai

    (The University of British Columbia)

  • Yanmei Lin

    (Sun Yat-sen University)

  • Wei Ke

    (Sun Yat-sen University)

  • Mustafa Hammood

    (The University of British Columbia)

  • Tianye Wang

    (The University of British Columbia)

  • Mengyue Xu

    (Sun Yat-sen University)

  • Zibo Zheng

    (Université Laval)

  • Mohammed Al-Qadasi

    (The University of British Columbia)

  • Omid Esmaeeli

    (The University of British Columbia)

  • Mohamed Rahim

    (National Research Council)

  • Grzegorz Pakulski

    (National Research Council)

  • Jens Schmid

    (National Research Council)

  • Pedro Barrios

    (National Research Council)

  • Weihong Jiang

    (National Research Council)

  • Hugh Morison

    (Queen’s University)

  • Matthew Mitchell

    (The University of British Columbia)

  • Xun Guan

    (Tsinghua University)

  • Nicolas A. F. Jaeger

    (The University of British Columbia)

  • Leslie A. Rusch

    (Université Laval)

  • Sudip Shekhar

    (The University of British Columbia)

  • Wei Shi

    (Université Laval)

  • Siyuan Yu

    (Sun Yat-sen University)

  • Xinlun Cai

    (Sun Yat-sen University)

  • Lukas Chrostowski

    (The University of British Columbia)

Abstract

Photonics offers a transformative approach to artificial intelligence (AI) and neuromorphic computing by enabling low-latency, high-speed, and energy-efficient computations. However, conventional photonic tensor cores face significant challenges in constructing large-scale photonic neuromorphic networks. Here, we propose a fully integrated photonic tensor core, consisting of only two thin-film lithium niobate (TFLN) modulators, a III-V laser, and a charge-integration photoreceiver. Despite its simple architecture, it is capable of implementing an entire layer of a neural network with a computational speed of 120 GOPS, while also allowing flexible adjustment of the number of inputs (fan-in) and outputs (fan-out). Our tensor core supports rapid in-situ training with a weight update speed of 60 GHz. Furthermore, it successfully classifies (supervised learning) and clusters (unsupervised learning) 112 × 112-pixel images through in-situ training. To enable in-situ training for clustering AI tasks, we offer a solution for performing multiplications between two negative numbers.

Suggested Citation

  • Zhongjin Lin & Bhavin J. Shastri & Shangxuan Yu & Jingxiang Song & Yuntao Zhu & Arman Safarnejadian & Wangning Cai & Yanmei Lin & Wei Ke & Mustafa Hammood & Tianye Wang & Mengyue Xu & Zibo Zheng & Moh, 2024. "120 GOPS Photonic tensor core in thin-film lithium niobate for inference and in situ training," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
  • Handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-53261-x
    DOI: 10.1038/s41467-024-53261-x
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-024-53261-x
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-024-53261-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. G. Mourgias-Alexandris & M. Moralis-Pegios & A. Tsakyridis & S. Simos & G. Dabos & A. Totovic & N. Passalis & M. Kirtas & T. Rutirawut & F. Y. Gardes & A. Tefas & N. Pleros, 2022. "Noise-resilient and high-speed deep learning with coherent silicon photonics," Nature Communications, Nature, vol. 13(1), pages 1-7, December.
    2. J. Feldmann & N. Youngblood & M. Karpov & H. Gehring & X. Li & M. Stappers & M. Gallo & X. Fu & A. Lukashchuk & A. S. Raja & J. Liu & C. D. Wright & A. Sebastian & T. J. Kippenberg & W. H. P. Pernice , 2021. "Publisher Correction: Parallel convolutional processing using an integrated photonic tensor core," Nature, Nature, vol. 591(7849), pages 13-13, March.
    3. Wen Zhou & Bowei Dong & Nikolaos Farmakidis & Xuan Li & Nathan Youngblood & Kairan Huang & Yuhan He & C. David Wright & Wolfram H. P. Pernice & Harish Bhaskaran, 2023. "In-memory photonic dot-product engine with electrically programmable weight banks," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    4. H. Zhang & M. Gu & X. D. Jiang & J. Thompson & H. Cai & S. Paesani & R. Santagati & A. Laing & Y. Zhang & M. H. Yung & Y. Z. Shi & F. K. Muhammad & G. Q. Lo & X. S. Luo & B. Dong & D. L. Kwong & L. C., 2021. "An optical neural chip for implementing complex-valued neural network," Nature Communications, Nature, vol. 12(1), pages 1-11, December.
    5. JOURNEE, Michel & NESTEROV, Yurii & RICHTARIK, Peter & SEPULCHRE, Rodolphe, 2010. "Generalized power method for sparse principal component analysis," LIDAM Reprints CORE 2232, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    6. Tingzhao Fu & Yubin Zang & Yuyao Huang & Zhenmin Du & Honghao Huang & Chengyang Hu & Minghua Chen & Sigang Yang & Hongwei Chen, 2023. "Photonic machine learning with on-chip diffractive optics," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    7. Xingyuan Xu & Mengxi Tan & Bill Corcoran & Jiayang Wu & Andreas Boes & Thach G. Nguyen & Sai T. Chu & Brent E. Little & Damien G. Hicks & Roberto Morandotti & Arnan Mitchell & David J. Moss, 2021. "11 TOPS photonic convolutional accelerator for optical neural networks," Nature, Nature, vol. 589(7840), pages 44-51, January.
    8. J. Feldmann & N. Youngblood & M. Karpov & H. Gehring & X. Li & M. Stappers & M. Gallo & X. Fu & A. Lukashchuk & A. S. Raja & J. Liu & C. D. Wright & A. Sebastian & T. J. Kippenberg & W. H. P. Pernice , 2021. "Parallel convolutional processing using an integrated photonic tensor core," Nature, Nature, vol. 589(7840), pages 52-58, January.
    9. Cheng Wang & Mian Zhang & Xi Chen & Maxime Bertrand & Amirhassan Shams-Ansari & Sethumadhavan Chandrasekhar & Peter Winzer & Marko Lončar, 2018. "Integrated lithium niobate electro-optic modulators operating at CMOS-compatible voltages," Nature, Nature, vol. 562(7725), pages 101-104, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Junwei Cheng & Chaoran Huang & Jialong Zhang & Bo Wu & Wenkai Zhang & Xinyu Liu & Jiahui Zhang & Yiyi Tang & Hailong Zhou & Qiming Zhang & Min Gu & Jianji Dong & Xinliang Zhang, 2024. "Multimodal deep learning using on-chip diffractive optics with in situ training capability," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    2. Xuan-Kun Li & Jian-Xu Ma & Xiang-Yu Li & Jun-Jie Hu & Chuan-Yang Ding & Feng-Kai Han & Xiao-Min Guo & Xi Tan & Xian-Min Jin, 2024. "High-efficiency reinforcement learning with hybrid architecture photonic integrated circuit," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    3. Han Zhao & Bingzhao Li & Huan Li & Mo Li, 2022. "Enabling scalable optical computing in synthetic frequency dimension using integrated cavity acousto-optics," Nature Communications, Nature, vol. 13(1), pages 1-7, December.
    4. Bowen Bai & Qipeng Yang & Haowen Shu & Lin Chang & Fenghe Yang & Bitao Shen & Zihan Tao & Jing Wang & Shaofu Xu & Weiqiang Xie & Weiwen Zou & Weiwei Hu & John E. Bowers & Xingjun Wang, 2023. "Microcomb-based integrated photonic processing unit," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    5. Chenduan Chen & Zhan Yang & Tao Wang & Yalun Wang & Kai Gao & Jiajia Wu & Jun Wang & Jianrong Qiu & Dezhi Tan, 2024. "Ultra-broadband all-optical nonlinear activation function enabled by MoTe2/optical waveguide integrated devices," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    6. Xiangyan Meng & Guojie Zhang & Nuannuan Shi & Guangyi Li & José Azaña & José Capmany & Jianping Yao & Yichen Shen & Wei Li & Ninghua Zhu & Ming Li, 2023. "Compact optical convolution processing unit based on multimode interference," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    7. Jingwei Ling & Zhengdong Gao & Shixin Xue & Qili Hu & Mingxiao Li & Kaibo Zhang & Usman A. Javid & Raymond Lopez-Rios & Jeremy Staffa & Qiang Lin, 2024. "Electrically empowered microcomb laser," Nature Communications, Nature, vol. 15(1), pages 1-8, December.
    8. Guangwei Cong & Noritsugu Yamamoto & Takashi Inoue & Yuriko Maegami & Morifumi Ohno & Shota Kita & Shu Namiki & Koji Yamada, 2022. "On-chip bacterial foraging training in silicon photonic circuits for projection-enabled nonlinear classification," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    9. Shaofu Xu & Jing Wang & Sicheng Yi & Weiwen Zou, 2022. "High-order tensor flow processing using integrated photonic circuits," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    10. G. Mourgias-Alexandris & M. Moralis-Pegios & A. Tsakyridis & S. Simos & G. Dabos & A. Totovic & N. Passalis & M. Kirtas & T. Rutirawut & F. Y. Gardes & A. Tefas & N. Pleros, 2022. "Noise-resilient and high-speed deep learning with coherent silicon photonics," Nature Communications, Nature, vol. 13(1), pages 1-7, December.
    11. Wen Zhou & Bowei Dong & Nikolaos Farmakidis & Xuan Li & Nathan Youngblood & Kairan Huang & Yuhan He & C. David Wright & Wolfram H. P. Pernice & Harish Bhaskaran, 2023. "In-memory photonic dot-product engine with electrically programmable weight banks," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    12. Chen-Guang Wang & Wuyue Xu & Chong Li & Lili Shi & Junliang Jiang & Tingting Guo & Wen-Cheng Yue & Tianyu Li & Ping Zhang & Yang-Yang Lyu & Jiazheng Pan & Xiuhao Deng & Ying Dong & Xuecou Tu & Sining , 2024. "Integrated and DC-powered superconducting microcomb," Nature Communications, Nature, vol. 15(1), pages 1-7, December.
    13. Seou Choi & Yannick Salamin & Charles Roques-Carmes & Rumen Dangovski & Di Luo & Zhuo Chen & Michael Horodynski & Jamison Sloan & Shiekh Zia Uddin & Marin Soljačić, 2024. "Photonic probabilistic machine learning using quantum vacuum noise," Nature Communications, Nature, vol. 15(1), pages 1-8, December.
    14. Wenting Wang & Ping-Keng Lu & Abhinav Kumar Vinod & Deniz Turan & James F. McMillan & Hao Liu & Mingbin Yu & Dim-Lee Kwong & Mona Jarrahi & Chee Wei Wong, 2022. "Coherent terahertz radiation with 2.8-octave tunability through chip-scale photomixed microresonator optical parametric oscillation," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    15. H. H. Zhu & J. Zou & H. Zhang & Y. Z. Shi & S. B. Luo & N. Wang & H. Cai & L. X. Wan & B. Wang & X. D. Jiang & J. Thompson & X. S. Luo & X. H. Zhou & L. M. Xiao & W. Huang & L. Patrick & M. Gu & L. C., 2022. "Space-efficient optical computing with an integrated chip diffractive neural network," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    16. Bitao Shen & Haowen Shu & Weiqiang Xie & Ruixuan Chen & Zhi Liu & Zhangfeng Ge & Xuguang Zhang & Yimeng Wang & Yunhao Zhang & Buwen Cheng & Shaohua Yu & Lin Chang & Xingjun Wang, 2023. "Harnessing microcomb-based parallel chaos for random number generation and optical decision making," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    17. Xuguang Zhang & Zixuan Zhou & Yijun Guo & Minxue Zhuang & Warren Jin & Bitao Shen & Yujun Chen & Jiahui Huang & Zihan Tao & Ming Jin & Ruixuan Chen & Zhangfeng Ge & Zhou Fang & Ning Zhang & Yadong Liu, 2024. "High-coherence parallelization in integrated photonics," Nature Communications, Nature, vol. 15(1), pages 1-9, December.
    18. Yiwei Li & Ning An & Zheyi Lu & Yuchen Wang & Bing Chang & Teng Tan & Xuhan Guo & Xizhen Xu & Jun He & Handing Xia & Zhaohui Wu & Yikai Su & Yuan Liu & Yunjiang Rao & Giancarlo Soavi & Baicheng Yao, 2022. "Nonlinear co-generation of graphene plasmons for optoelectronic logic operations," Nature Communications, Nature, vol. 13(1), pages 1-7, December.
    19. Mitsumasa Nakajima & Katsuma Inoue & Kenji Tanaka & Yasuo Kuniyoshi & Toshikazu Hashimoto & Kohei Nakajima, 2022. "Physical deep learning with biologically inspired training method: gradient-free approach for physical hardware," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    20. Miltiadis Moralis-Pegios & George Giamougiannis & Apostolos Tsakyridis & David Lazovsky & Nikos Pleros, 2024. "Perfect linear optics using silicon photonics," Nature Communications, Nature, vol. 15(1), pages 1-8, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-53261-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.