IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0219004.html
   My bibliography  Save this article

BioGD: Bio-inspired robust gradient descent

Author

Listed:
  • Ilona Kulikovskikh
  • Sergej Prokhorov
  • Tomislav Lipić
  • Tarzan Legović
  • Tomislav Šmuc

Abstract

Recent research in machine learning pointed to the core problem of state-of-the-art models which impedes their widespread adoption in different domains. The models’ inability to differentiate between noise and subtle, yet significant variation in data leads to their vulnerability to adversarial perturbations that cause wrong predictions with high confidence. The study is aimed at identifying whether the algorithms inspired by biological evolution may achieve better results in cases where brittle robustness properties are highly sensitive to the slight noise. To answer this question, we introduce the new robust gradient descent inspired by the stability and adaptability of biological systems to unknown and changing environments. The proposed optimization technique involves an open-ended adaptation process with regard to two hyperparameters inherited from the generalized Verhulst population growth equation. The hyperparameters increase robustness to adversarial noise by penalizing the degree to which hardly visible changes in gradients impact prediction. The empirical evidence on synthetic and experimental datasets confirmed the viability of the bio-inspired gradient descent and suggested promising directions for future research. The code used for computational experiments is provided in a repository at https://github.com/yukinoi/bio_gradient_descent.

Suggested Citation

  • Ilona Kulikovskikh & Sergej Prokhorov & Tomislav Lipić & Tarzan Legović & Tomislav Šmuc, 2019. "BioGD: Bio-inspired robust gradient descent," PLOS ONE, Public Library of Science, vol. 14(7), pages 1-19, July.
  • Handle: RePEc:plo:pone00:0219004
    DOI: 10.1371/journal.pone.0219004
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0219004
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0219004&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0219004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. James Dawes, 2017. "The case for and against autonomous weapon systems," Nature Human Behaviour, Nature, vol. 1(9), pages 613-614, September.
    2. Amy Maxmen, 2018. "Deep learning sharpens views of cells and genes," Nature, Nature, vol. 553(7686), pages 9-10, January.
    3. Dario Floreano & Robert J. Wood, 2015. "Science, technology and the future of small autonomous drones," Nature, Nature, vol. 521(7553), pages 460-466, May.
    4. Kevin M. Cherry & Lulu Qian, 2018. "Scaling up molecular pattern recognition with DNA-based winner-take-all neural networks," Nature, Nature, vol. 559(7714), pages 370-376, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Qian Li & Ting Tan & Benlong Wang & Zhimiao Yan, 2024. "Avian-inspired embodied perception in biohybrid flapping-wing robotics," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    2. Stöcker, Claudia & Bennett, Rohan & Koeva, Mila & Nex, Francesco & Zevenbergen, Jaap, 2022. "Scaling up UAVs for land administration: Towards the plateau of productivity," Land Use Policy, Elsevier, vol. 114(C).
    3. Zenan Shen & Shaoquan Liu & Wei Zhu & Daoyuan Ren & Qiang Xu & Yu Feng, 2024. "A Review on Key Technologies and Developments of Hydrogen Fuel Cell Multi-Rotor Drones," Energies, MDPI, vol. 17(16), pages 1-36, August.
    4. Kloster, Konstantin & Moeini, Mahdi & Vigo, Daniele & Wendt, Oliver, 2023. "The multiple traveling salesman problem in presence of drone- and robot-supported packet stations," European Journal of Operational Research, Elsevier, vol. 305(2), pages 630-643.
    5. Linlin Yang & Qian Tang & Mingzhi Zhang & Yuan Tian & Xiaoxing Chen & Rui Xu & Qian Ma & Pei Guo & Chao Zhang & Da Han, 2024. "A spatially localized DNA linear classifier for cancer diagnosis," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    6. Gabriel de Freitas Viscondi & Solange N. Alves-Souza, 2021. "Solar Irradiance Prediction with Machine Learning Algorithms: A Brazilian Case Study on Photovoltaic Electricity Generation," Energies, MDPI, vol. 14(18), pages 1-15, September.
    7. Zhu, Xun & Pasch, Timothy J. & Bergstrom, Aaron, 2020. "Understanding the structure of risk belief systems concerning drone delivery: A network analysis," Technology in Society, Elsevier, vol. 62(C).
    8. Yandong Xiao & Xiaokang Lei & Zhicheng Zheng & Yalun Xiang & Yang-Yu Liu & Xingguang Peng, 2024. "Perception of motion salience shapes the emergence of collective motions," Nature Communications, Nature, vol. 15(1), pages 1-16, December.
    9. Kai Fukami & Kunihiko Taira, 2023. "Grasping extreme aerodynamics on a low-dimensional manifold," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    10. Gonzalo Fernandez-Sanchez & Alvaro Fernandez-Heredia, 2018. "Strategic Thinking for Sustainability: A Review of 10 Strategies for Sustainable Mobility by Bus for Cities," Sustainability, MDPI, vol. 10(11), pages 1-15, November.
    11. Karen Zhang & Yuan-Jyue Chen & Delaney Wilde & Kathryn Doroschak & Karin Strauss & Luis Ceze & Georg Seelig & Jeff Nivala, 2022. "A nanopore interface for higher bandwidth DNA computing," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    12. Hongbo He & Xiaohan Liao & Huping Ye & Chenchen Xu & Huanyin Yue, 2023. "Data-Driven Insights into Population Exposure Risks: Towards Sustainable and Safe Urban Airspace Utilization by Unmanned Aerial Systems," Sustainability, MDPI, vol. 15(16), pages 1-20, August.
    13. Schniederjans, Dara G. & Curado, Carla & Khalajhedayati, Mehrnaz, 2020. "Supply chain digitisation trends: An integration of knowledge management," International Journal of Production Economics, Elsevier, vol. 220(C).
    14. Jianwei Sun & Koichi Yonezawa & Eiji Shima & Hao Liu, 2023. "Integrated Evaluation of the Aeroacoustics and Psychoacoustics of a Single Propeller," IJERPH, MDPI, vol. 20(3), pages 1-23, January.
    15. Christian Wankmüller & Christian Truden & Christopher Korzen & Philipp Hungerländer & Ewald Kolesnik & Gerald Reiner, 2020. "Optimal allocation of defibrillator drones in mountainous regions," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 42(3), pages 785-814, September.
    16. Ghada Talat Alhothali & Felix T. Mavondo & Bader A. Alyoubi & Haneen Algethami, 2024. "Consumer Acceptance of Drones for Last-Mile Delivery in Jeddah, Saudi Arabia," Sustainability, MDPI, vol. 16(13), pages 1-21, June.
    17. David Reiser & Galibjon M. Sharipov & Gero Hubel & Volker Nannen & Hans W. Griepentrog, 2023. "Development and Experimental Validation of an Agricultural Robotic Platform with High Traction and Low Compaction," Agriculture, MDPI, vol. 13(8), pages 1-15, July.
    18. Kanakov, Oleg & Chen, Shangbin & Zaikin, Alexey, 2024. "Learning by selective plasmid loss for intracellular synthetic classifiers," Chaos, Solitons & Fractals, Elsevier, vol. 179(C).
    19. Köbis, Nils & Rahwan, Zoe & Bersch, Clara & Ajaj, Tamer & Bonnefon, Jean-François & Rahwan, Iyad, 2024. "Experimental evidence that delegating to intelligent machines can increase dishonest behaviour," OSF Preprints dnjgz, Center for Open Science.
    20. Wang, Ning & Mutzner, Nico & Blanchet, Karl, 2023. "Societal acceptance of urban drones: A scoping literature review," Technology in Society, Elsevier, vol. 75(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0219004. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.