IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v12y2021i1d10.1038_s41467-021-25874-z.html
   My bibliography  Save this article

Embodied intelligence via learning and evolution

Author

Listed:
  • Agrim Gupta

    (Stanford University)

  • Silvio Savarese

    (Stanford University)

  • Surya Ganguli

    (Stanford University
    Stanford University
    Stanford University)

  • Li Fei-Fei

    (Stanford University
    Stanford University)

Abstract

The intertwined processes of learning and evolution in complex environmental niches have resulted in a remarkable diversity of morphological forms. Moreover, many aspects of animal intelligence are deeply embodied in these evolved morphologies. However, the principles governing relations between environmental complexity, evolved morphology, and the learnability of intelligent control, remain elusive, because performing large-scale in silico experiments on evolution and learning is challenging. Here, we introduce Deep Evolutionary Reinforcement Learning (DERL): a computational framework which can evolve diverse agent morphologies to learn challenging locomotion and manipulation tasks in complex environments. Leveraging DERL we demonstrate several relations between environmental complexity, morphological intelligence and the learnability of control. First, environmental complexity fosters the evolution of morphological intelligence as quantified by the ability of a morphology to facilitate the learning of novel tasks. Second, we demonstrate a morphological Baldwin effect i.e., in our simulations evolution rapidly selects morphologies that learn faster, thereby enabling behaviors learned late in the lifetime of early ancestors to be expressed early in the descendants lifetime. Third, we suggest a mechanistic basis for the above relationships through the evolution of morphologies that are more physically stable and energy efficient, and can therefore facilitate learning and control.

Suggested Citation

  • Agrim Gupta & Silvio Savarese & Surya Ganguli & Li Fei-Fei, 2021. "Embodied intelligence via learning and evolution," Nature Communications, Nature, vol. 12(1), pages 1-12, December.
  • Handle: RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-25874-z
    DOI: 10.1038/s41467-021-25874-z
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-021-25874-z
    File Function: Abstract
    Download Restriction: no

    File URL: https://libkey.io/10.1038/s41467-021-25874-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Hod Lipson & Jordan B. Pollack, 2000. "Automatic design and manufacture of robotic lifeforms," Nature, Nature, vol. 406(6799), pages 974-978, August.
    2. Daniela Rus & Michael T. Tolley, 2015. "Design, fabrication and control of soft robots," Nature, Nature, vol. 521(7553), pages 467-475, May.
    3. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Anthony Zador & Sean Escola & Blake Richards & Bence Ölveczky & Yoshua Bengio & Kwabena Boahen & Matthew Botvinick & Dmitri Chklovskii & Anne Churchland & Claudia Clopath & James DiCarlo & Surya Gangu, 2023. "Catalyzing next-generation Artificial Intelligence through NeuroAI," Nature Communications, Nature, vol. 14(1), pages 1-7, December.
    2. Xiao-Yang Liu & Ziyi Xia & Jingyang Rui & Jiechao Gao & Hongyang Yang & Ming Zhu & Christina Dan Wang & Zhaoran Wang & Jian Guo, 2022. "FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning," Papers 2211.03107, arXiv.org.
    3. Xiao-Yang Liu & Jingyang Rui & Jiechao Gao & Liuqing Yang & Hongyang Yang & Zhaoran Wang & Christina Dan Wang & Jian Guo, 2021. "FinRL-Meta: A Universe of Near-Real Market Environments for Data-Driven Deep Reinforcement Learning in Quantitative Finance," Papers 2112.06753, arXiv.org, revised Mar 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Girish Chowdhary & Mattia Gazzola & Girish Krishnan & Chinmay Soman & Sarah Lovell, 2019. "Soft Robotics as an Enabling Technology for Agroforestry Practice and Research," Sustainability, MDPI, vol. 11(23), pages 1-21, November.
    2. Xiaoyue Li & John M. Mulvey, 2023. "Optimal Portfolio Execution in a Regime-switching Market with Non-linear Impact Costs: Combining Dynamic Program and Neural Network," Papers 2306.08809, arXiv.org.
    3. Nathan Companez & Aldeida Aleti, 2016. "Can Monte-Carlo Tree Search learn to sacrifice?," Journal of Heuristics, Springer, vol. 22(6), pages 783-813, December.
    4. Shijing Zhang & Yingxiang Liu & Jie Deng & Xiang Gao & Jing Li & Weiyi Wang & Mingxin Xun & Xuefeng Ma & Qingbing Chang & Junkao Liu & Weishan Chen & Jie Zhao, 2023. "Piezo robotic hand for motion manipulation from micro to macro," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    5. Amir Souhail & Passakorn vassakosol, 2018. "Low Cost Soft Robotic Grippers For Reliable Grasping," Journal of Mechanical Engineering Research & Developments (JMERD), Zibeline International Publishing, vol. 41(4), pages 88-95, November.
    6. Zhewei Zhang & Youngjin Yoo & Kalle Lyytinen & Aron Lindberg, 2021. "The Unknowability of Autonomous Tools and the Liminal Experience of Their Use," Information Systems Research, INFORMS, vol. 32(4), pages 1192-1213, December.
    7. Yuhong Wang & Lei Chen & Hong Zhou & Xu Zhou & Zongsheng Zheng & Qi Zeng & Li Jiang & Liang Lu, 2021. "Flexible Transmission Network Expansion Planning Based on DQN Algorithm," Energies, MDPI, vol. 14(7), pages 1-21, April.
    8. Gokhale, Gargya & Claessens, Bert & Develder, Chris, 2022. "Physics informed neural networks for control oriented thermal modeling of buildings," Applied Energy, Elsevier, vol. 314(C).
    9. Li Xia, 2020. "Risk‐Sensitive Markov Decision Processes with Combined Metrics of Mean and Variance," Production and Operations Management, Production and Operations Management Society, vol. 29(12), pages 2808-2827, December.
    10. Neha Soni & Enakshi Khular Sharma & Narotam Singh & Amita Kapoor, 2019. "Impact of Artificial Intelligence on Businesses: from Research, Innovation, Market Deployment to Future Shifts in Business Models," Papers 1905.02092, arXiv.org.
    11. Yin, Linfei & He, Xiaoyu, 2023. "Artificial emotional deep Q learning for real-time smart voltage control of cyber-physical social power systems," Energy, Elsevier, vol. 273(C).
    12. Taejong Joo & Hyunyoung Jun & Dongmin Shin, 2022. "Task Allocation in Human–Machine Manufacturing Systems Using Deep Reinforcement Learning," Sustainability, MDPI, vol. 14(4), pages 1-18, February.
    13. Burka, Dávid & Puppe, Clemens & Szepesváry, László & Tasnádi, Attila, 2022. "Voting: A machine learning approach," European Journal of Operational Research, Elsevier, vol. 299(3), pages 1003-1017.
    14. Oleh Lukianykhin & Tetiana Bogodorova, 2021. "Voltage Control-Based Ancillary Service Using Deep Reinforcement Learning," Energies, MDPI, vol. 14(8), pages 1-22, April.
    15. De Bruyn, Arnaud & Viswanathan, Vijay & Beh, Yean Shan & Brock, Jürgen Kai-Uwe & von Wangenheim, Florian, 2020. "Artificial Intelligence and Marketing: Pitfalls and Opportunities," Journal of Interactive Marketing, Elsevier, vol. 51(C), pages 91-105.
    16. Keerthana Sivamayil & Elakkiya Rajasekar & Belqasem Aljafari & Srete Nikolovski & Subramaniyaswamy Vairavasundaram & Indragandhi Vairavasundaram, 2023. "A Systematic Study on Reinforcement Learning Based Applications," Energies, MDPI, vol. 16(3), pages 1-23, February.
    17. Chen, Jiaxin & Shu, Hong & Tang, Xiaolin & Liu, Teng & Wang, Weida, 2022. "Deep reinforcement learning-based multi-objective control of hybrid power system combined with road recognition under time-varying environment," Energy, Elsevier, vol. 239(PC).
    18. Amirhosein Mosavi & Yaser Faghan & Pedram Ghamisi & Puhong Duan & Sina Faizollahzadeh Ardabili & Ely Salwana & Shahab S. Band, 2020. "Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics," Mathematics, MDPI, vol. 8(10), pages 1-42, September.
    19. Zhang, Yihao & Chai, Zhaojie & Lykotrafitis, George, 2021. "Deep reinforcement learning with a particle dynamics environment applied to emergency evacuation of a room with obstacles," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    20. Haoran Wang & Xun Yu Zhou, 2020. "Continuous‐time mean–variance portfolio selection: A reinforcement learning framework," Mathematical Finance, Wiley Blackwell, vol. 30(4), pages 1273-1308, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-25874-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.