IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v16y2025i1d10.1038_s41467-025-56254-6.html
   My bibliography  Save this article

Efficient nonlinear function approximation in analog resistive crossbars for recurrent neural networks

Author

Listed:
  • Junyi Yang

    (City University of Hong Kong)

  • Ruibin Mao

    (The University of Hong Kong)

  • Mingrui Jiang

    (The University of Hong Kong)

  • Yichuan Cheng

    (City University of Hong Kong)

  • Pao-Sheng Vincent Sun

    (City University of Hong Kong)

  • Shuai Dong

    (City University of Hong Kong)

  • Giacomo Pedretti

    (Hewlett Packard Enterprise)

  • Xia Sheng

    (Hewlett Packard Enterprise)

  • Jim Ignowski

    (Hewlett Packard Enterprise)

  • Haoliang Li

    (City University of Hong Kong)

  • Can Li

    (The University of Hong Kong)

  • Arindam Basu

    (City University of Hong Kong)

Abstract

Analog In-memory Computing (IMC) has demonstrated energy-efficient and low latency implementation of convolution and fully-connected layers in deep neural networks (DNN) by using physics for computing in parallel resistive memory arrays. However, recurrent neural networks (RNN) that are widely used for speech-recognition and natural language processing have tasted limited success with this approach. This can be attributed to the significant time and energy penalties incurred in implementing nonlinear activation functions that are abundant in such models. In this work, we experimentally demonstrate the implementation of a non-linear activation function integrated with a ramp analog-to-digital conversion (ADC) at the periphery of the memory to improve in-memory implementation of RNNs. Our approach uses an extra column of memristors to produce an appropriately pre-distorted ramp voltage such that the comparator output directly approximates the desired nonlinear function. We experimentally demonstrate programming different nonlinear functions using a memristive array and simulate its incorporation in RNNs to solve keyword spotting and language modelling tasks. Compared to other approaches, we demonstrate manifold increase in area-efficiency, energy-efficiency and throughput due to the in-memory, programmable ramp generator that removes digital processing overhead.

Suggested Citation

  • Junyi Yang & Ruibin Mao & Mingrui Jiang & Yichuan Cheng & Pao-Sheng Vincent Sun & Shuai Dong & Giacomo Pedretti & Xia Sheng & Jim Ignowski & Haoliang Li & Can Li & Arindam Basu, 2025. "Efficient nonlinear function approximation in analog resistive crossbars for recurrent neural networks," Nature Communications, Nature, vol. 16(1), pages 1-15, December.
  • Handle: RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-56254-6
    DOI: 10.1038/s41467-025-56254-6
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-025-56254-6
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-025-56254-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. S. Ambrogio & P. Narayanan & A. Okazaki & A. Fasoli & C. Mackin & K. Hosokawa & A. Nomura & T. Yasuda & A. Chen & A. Friz & M. Ishii & J. Luquin & Y. Kohda & N. Saulnier & K. Brew & S. Choi & I. Ok & , 2023. "An analog-AI chip for energy-efficient speech recognition and transcription," Nature, Nature, vol. 620(7975), pages 768-775, August.
    2. M. Prezioso & F. Merrikh-Bayat & B. D. Hoskins & G. C. Adam & K. K. Likharev & D. B. Strukov, 2015. "Training and operation of an integrated neuromorphic network based on metal-oxide memristors," Nature, Nature, vol. 521(7550), pages 61-64, May.
    3. Mingyi Rao & Hao Tang & Jiangbin Wu & Wenhao Song & Max Zhang & Wenbo Yin & Ye Zhuo & Fatemeh Kiani & Benjamin Chen & Xiangqi Jiang & Hefei Liu & Hung-Yu Chen & Rivu Midya & Fan Ye & Hao Jiang & Zhong, 2023. "Thousands of conductance levels in memristors integrated on CMOS," Nature, Nature, vol. 615(7954), pages 823-829, March.
    4. Andrew W. Senior & Richard Evans & John Jumper & James Kirkpatrick & Laurent Sifre & Tim Green & Chongli Qin & Augustin Žídek & Alexander W. R. Nelson & Alex Bridgland & Hugo Penedones & Stig Petersen, 2020. "Improved protein structure prediction using potentials from deep learning," Nature, Nature, vol. 577(7792), pages 706-710, January.
    5. Ruibin Mao & Bo Wen & Arman Kazemi & Yahui Zhao & Ann Franchesca Laguna & Rui Lin & Ngai Wong & Michael Niemier & X. Sharon Hu & Xia Sheng & Catherine E. Graves & John Paul Strachan & Can Li, 2022. "Experimentally validated memristive memory augmented neural network with efficient hashing and similarity search," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    6. Junyi Yang & Ruibin Mao & Mingrui Jiang & Yichuan Cheng & Pao-Sheng Vincent Sun & Shuai Dong & Giacomo Pedretti & Xia Sheng & Jim Ignowski & Haoliang Li & Can Li & Arindam Basu, 2025. "Efficient nonlinear function approximation in analog resistive crossbars for recurrent neural networks," Nature Communications, Nature, vol. 16(1), pages 1-15, December.
    7. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Junyi Yang & Ruibin Mao & Mingrui Jiang & Yichuan Cheng & Pao-Sheng Vincent Sun & Shuai Dong & Giacomo Pedretti & Xia Sheng & Jim Ignowski & Haoliang Li & Can Li & Arindam Basu, 2025. "Efficient nonlinear function approximation in analog resistive crossbars for recurrent neural networks," Nature Communications, Nature, vol. 16(1), pages 1-15, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Thomas Ortner & Horst Petschenig & Athanasios Vasilopoulos & Roland Renner & Špela Brglez & Thomas Limbacher & Enrique Piñero & Alejandro Linares-Barranco & Angeliki Pantazi & Robert Legenstein, 2025. "Rapid learning with phase-change memory-based in-memory computing through learning-to-learn," Nature Communications, Nature, vol. 16(1), pages 1-16, December.
    2. Yijun Li & Jianshi Tang & Bin Gao & Jian Yao & Anjunyi Fan & Bonan Yan & Yuchao Yang & Yue Xi & Yuankun Li & Jiaming Li & Wen Sun & Yiwei Du & Zhengwu Liu & Qingtian Zhang & Song Qiu & Qingwen Li & He, 2023. "Monolithic three-dimensional integration of RRAM-based hybrid memory architecture for one-shot learning," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    3. Liu, Bokai & Wang, Yizheng & Rabczuk, Timon & Olofsson, Thomas & Lu, Weizhuo, 2024. "Multi-scale modeling in thermal conductivity of Polyurethane incorporated with Phase Change Materials using Physics-Informed Neural Networks," Renewable Energy, Elsevier, vol. 220(C).
    4. Peng Chen & Fenghao Liu & Peng Lin & Peihong Li & Yu Xiao & Bihua Zhang & Gang Pan, 2023. "Open-loop analog programmable electrochemical memory array," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    5. Tian Lan & Huan Wang & Qi An, 2024. "Enabling high throughput deep reinforcement learning with first principles to investigate catalytic reaction mechanisms," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    6. Thomas Dalgaty & Filippo Moro & Yiğit Demirağ & Alessio Pra & Giacomo Indiveri & Elisa Vianello & Melika Payvand, 2024. "Mosaic: in-memory computing and routing for small-world spike-based neuromorphic systems," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    7. Pietro Belleri & Judith Pons i Tarrés & Iain McCulloch & Paul W. M. Blom & Zsolt M. Kovács-Vajna & Paschalis Gkoupidenis & Fabrizio Torricelli, 2024. "Unravelling the operation of organic artificial neurons for neuromorphic bioelectronics," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    8. Yuyan Zhu & Yang Wang & Xingchen Pang & Yongbo Jiang & Xiaoxian Liu & Qing Li & Zhen Wang & Chunsen Liu & Weida Hu & Peng Zhou, 2024. "Non-volatile 2D MoS2/black phosphorus heterojunction photodiodes in the near- to mid-infrared region," Nature Communications, Nature, vol. 15(1), pages 1-10, December.
    9. Xiaoyue Li & John M. Mulvey, 2023. "Optimal Portfolio Execution in a Regime-switching Market with Non-linear Impact Costs: Combining Dynamic Program and Neural Network," Papers 2306.08809, arXiv.org.
    10. Nathan Companez & Aldeida Aleti, 2016. "Can Monte-Carlo Tree Search learn to sacrifice?," Journal of Heuristics, Springer, vol. 22(6), pages 783-813, December.
    11. Zhewei Zhang & Youngjin Yoo & Kalle Lyytinen & Aron Lindberg, 2021. "The Unknowability of Autonomous Tools and the Liminal Experience of Their Use," Information Systems Research, INFORMS, vol. 32(4), pages 1192-1213, December.
    12. Yuhong Wang & Lei Chen & Hong Zhou & Xu Zhou & Zongsheng Zheng & Qi Zeng & Li Jiang & Liang Lu, 2021. "Flexible Transmission Network Expansion Planning Based on DQN Algorithm," Energies, MDPI, vol. 14(7), pages 1-21, April.
    13. Gokhale, Gargya & Claessens, Bert & Develder, Chris, 2022. "Physics informed neural networks for control oriented thermal modeling of buildings," Applied Energy, Elsevier, vol. 314(C).
    14. Lauren L. Porter & Allen K. Kim & Swechha Rimal & Loren L. Looger & Ananya Majumdar & Brett D. Mensh & Mary R. Starich & Marie-Paule Strub, 2022. "Many dissimilar NusG protein domains switch between α-helix and β-sheet folds," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    15. Li Xia, 2020. "Risk‐Sensitive Markov Decision Processes with Combined Metrics of Mean and Variance," Production and Operations Management, Production and Operations Management Society, vol. 29(12), pages 2808-2827, December.
    16. Neha Soni & Enakshi Khular Sharma & Narotam Singh & Amita Kapoor, 2019. "Impact of Artificial Intelligence on Businesses: from Research, Innovation, Market Deployment to Future Shifts in Business Models," Papers 1905.02092, arXiv.org.
    17. Yin, Linfei & He, Xiaoyu, 2023. "Artificial emotional deep Q learning for real-time smart voltage control of cyber-physical social power systems," Energy, Elsevier, vol. 273(C).
    18. Taejong Joo & Hyunyoung Jun & Dongmin Shin, 2022. "Task Allocation in Human–Machine Manufacturing Systems Using Deep Reinforcement Learning," Sustainability, MDPI, vol. 14(4), pages 1-18, February.
    19. Djohan Bonnet & Tifenn Hirtzlin & Atreya Majumdar & Thomas Dalgaty & Eduardo Esmanhotto & Valentina Meli & Niccolo Castellani & Simon Martin & Jean-François Nodin & Guillaume Bourgeois & Jean-Michel P, 2023. "Bringing uncertainty quantification to the extreme-edge with memristor-based Bayesian neural networks," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    20. Zachary C. Drake & Justin T. Seffernick & Steffen Lindert, 2022. "Protein complex prediction using Rosetta, AlphaFold, and mass spectrometry covalent labeling," Nature Communications, Nature, vol. 13(1), pages 1-9, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:16:y:2025:i:1:d:10.1038_s41467-025-56254-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.