IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i5p1183-d1082998.html
   My bibliography  Save this article

Modification of Learning Ratio and Drop-Out for Stochastic Gradient Descendant Algorithm

Author

Listed:
  • Adrian Teso-Fz-Betoño

    (System Engineering and Automation Control Department, University of the Basque Country (UPV/EHU), Nieves Cano, 12, 01006 Vitoria-Gasteiz, Spain)

  • Ekaitz Zulueta

    (System Engineering and Automation Control Department, University of the Basque Country (UPV/EHU), Nieves Cano, 12, 01006 Vitoria-Gasteiz, Spain)

  • Mireya Cabezas-Olivenza

    (System Engineering and Automation Control Department, University of the Basque Country (UPV/EHU), Nieves Cano, 12, 01006 Vitoria-Gasteiz, Spain)

  • Unai Fernandez-Gamiz

    (Department of Nuclear and Fluid Mechanics, University of the Basque Country (UPV/EHU), 01006 Vitoria-Gasteiz, Spain)

  • Carlos Botana-M-Ibarreta

    (System Engineering and Automation Control Department, University of the Basque Country (UPV/EHU), Nieves Cano, 12, 01006 Vitoria-Gasteiz, Spain)

Abstract

The stochastic gradient descendant algorithm is one of the most popular neural network training algorithms. Many authors have contributed to modifying or adapting its shape and parametrizations in order to improve its performance. In this paper, the authors propose two modifications on this algorithm that can result in a better performance without increasing significantly the computational and time resources needed. The first one is a dynamic learning ratio depending on the network layer where it is applied, and the second one is a dynamic drop-out that decreases through the epochs of training. These techniques have been tested against different benchmark function to see their effect on the learning process. The obtained results show that the application of these techniques improves the performance of the learning of the neural network, especially when they are used together.

Suggested Citation

  • Adrian Teso-Fz-Betoño & Ekaitz Zulueta & Mireya Cabezas-Olivenza & Unai Fernandez-Gamiz & Carlos Botana-M-Ibarreta, 2023. "Modification of Learning Ratio and Drop-Out for Stochastic Gradient Descendant Algorithm," Mathematics, MDPI, vol. 11(5), pages 1-24, February.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:5:p:1183-:d:1082998
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/5/1183/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/5/1183/
    Download Restriction: no
    ---><---

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:5:p:1183-:d:1082998. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.