IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i16p3019-d894277.html
   My bibliography  Save this article

A Hybrid Sparrow Search Algorithm of the Hyperparameter Optimization in Deep Learning

Author

Listed:
  • Yanyan Fan

    (School of Mechanical Engineering, Yanshan University, Qinhuangdao 066004, China)

  • Yu Zhang

    (School of Mechanical Engineering, Yanshan University, Qinhuangdao 066004, China)

  • Baosu Guo

    (School of Mechanical Engineering, Yanshan University, Qinhuangdao 066004, China)

  • Xiaoyuan Luo

    (School of Electrical Engineering, Yanshan University, Qinhuangdao 066004, China)

  • Qingjin Peng

    (Department of Mechanical Engineering, University of Manitoba, Winnipeg, MB R3T 5V6, Canada)

  • Zhenlin Jin

    (School of Mechanical Engineering, Yanshan University, Qinhuangdao 066004, China)

Abstract

Deep learning has been widely used in different fields such as computer vision and speech processing. The performance of deep learning algorithms is greatly affected by their hyperparameters. For complex machine learning models such as deep neural networks, it is difficult to determine their hyperparameters. In addition, existing hyperparameter optimization algorithms easily converge to a local optimal solution. This paper proposes a method for hyperparameter optimization that combines the Sparrow Search Algorithm and Particle Swarm Optimization, called the Hybrid Sparrow Search Algorithm. This method takes advantages of avoiding the local optimal solution in the Sparrow Search Algorithm and the search efficiency of Particle Swarm Optimization to achieve global optimization. Experiments verified the proposed algorithm in simple and complex networks. The results show that the Hybrid Sparrow Search Algorithm has the strong global search capability to avoid local optimal solutions and satisfactory search efficiency in both low and high-dimensional spaces. The proposed method provides a new solution for hyperparameter optimization problems in deep learning models.

Suggested Citation

  • Yanyan Fan & Yu Zhang & Baosu Guo & Xiaoyuan Luo & Qingjin Peng & Zhenlin Jin, 2022. "A Hybrid Sparrow Search Algorithm of the Hyperparameter Optimization in Deep Learning," Mathematics, MDPI, vol. 10(16), pages 1-23, August.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:16:p:3019-:d:894277
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/16/3019/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/16/3019/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mohamed Omri & Sayed Abdel-Khalek & Eied M. Khalil & Jamel Bouslimi & Gyanendra Prasad Joshi, 2022. "Modeling of Hyperparameter Tuned Deep Learning Model for Automated Image Captioning," Mathematics, MDPI, vol. 10(3), pages 1-20, January.
    2. P. M. Lerman, 1980. "Fitting Segmented Regression Models by Grid Search," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 29(1), pages 77-84, March.
    3. Andrey Gorshenin & Victor Kuzmin, 2022. "Statistical Feature Construction for Forecasting Accuracy Increase and Its Applications in Neural Network Based Analysis," Mathematics, MDPI, vol. 10(4), pages 1-21, February.
    4. Bruno Gašperov & Stjepan Begušić & Petra Posedel Šimović & Zvonko Kostanjčar, 2021. "Reinforcement Learning Approaches to Optimal Market Making," Mathematics, MDPI, vol. 9(21), pages 1-22, October.
    5. Mohammad Khishe & Fabio Caraffini & Stefan Kuhn, 2021. "Evolving Deep Learning Convolutional Neural Networks for Early COVID-19 Detection in Chest X-ray Images," Mathematics, MDPI, vol. 9(9), pages 1-18, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bonfiglio, A. & Camaioni, B. & Carta, V. & Cristiano, S., 2023. "Estimating the common agricultural policy milestones and targets by neural networks," Evaluation and Program Planning, Elsevier, vol. 99(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fan, Xudong & Wang, Xiaowei & Zhang, Xijin & ASCE Xiong (Bill) Yu, P.E.F., 2022. "Machine learning based water pipe failure prediction: The effects of engineering, geology, climate and socio-economic factors," Reliability Engineering and System Safety, Elsevier, vol. 219(C).
    2. Adham Alsharkawi & Mohammad Al-Fetyani & Maha Dawas & Heba Saadeh & Musa Alyaman, 2021. "Poverty Classification Using Machine Learning: The Case of Jordan," Sustainability, MDPI, vol. 13(3), pages 1-16, January.
    3. Ben Q. Liu & Dale L. Goodhue, 2012. "Two Worlds of Trust for Potential E-Commerce Users: Humans as Cognitive Misers," Information Systems Research, INFORMS, vol. 23(4), pages 1246-1262, December.
    4. Chen Huann-Sheng & Zeichner Sarah & Anderson Robert N. & Espey David K. & Kim Hyune-Ju & Feuer Eric J., 2020. "The Joinpoint-Jump and Joinpoint-Comparability Ratio Model for Trend Analysis with Applications to Coding Changes in Health Statistics," Journal of Official Statistics, Sciendo, vol. 36(1), pages 49-62, March.
    5. Tan, Xiujie & Xiao, Ziwei & Liu, Yishuang & Taghizadeh-Hesary, Farhad & Wang, Banban & Dong, Hanmin, 2022. "The effect of green credit policy on energy efficiency: Evidence from China," Technological Forecasting and Social Change, Elsevier, vol. 183(C).
    6. Muñoz, J.F. & Arcos, A. & Álvarez, E. & Rueda, M., 2014. "New ratio and difference estimators of the finite population distribution function," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 102(C), pages 51-61.
    7. Yong-Hyuk Kim & Fabio Caraffini, 2023. "Preface to “Swarm and Evolutionary Computation—Bridging Theory and Practice”," Mathematics, MDPI, vol. 11(5), pages 1-3, March.
    8. Suwon Song & Chun Gun Park, 2019. "Alternative Algorithm for Automatically Driving Best-Fit Building Energy Baseline Models Using a Data—Driven Grid Search," Sustainability, MDPI, vol. 11(24), pages 1-11, December.
    9. Yu, Binbing & Barrett, Michael J. & Kim, Hyune-Ju & Feuer, Eric J., 2007. "Estimating joinpoints in continuous time scale for multiple change-point models," Computational Statistics & Data Analysis, Elsevier, vol. 51(5), pages 2420-2427, February.
    10. Tahira Kootbodien & Nisha Naicker & Kerry S. Wilson & Raj Ramesar & Leslie London, 2020. "Trends in Suicide Mortality in South Africa, 1997 to 2016," IJERPH, MDPI, vol. 17(6), pages 1-16, March.
    11. Jonathan Readshaw & Stefano Giani, 2020. "Using Company Specific Headlines and Convolutional Neural Networks to Predict Stock Fluctuations," Papers 2006.12426, arXiv.org.
    12. Erjia Ge & Yee Leung, 2013. "Detection of crossover time scales in multifractal detrended fluctuation analysis," Journal of Geographical Systems, Springer, vol. 15(2), pages 115-147, April.
    13. Bucarey, Víctor & Labbé, Martine & Morales, Juan M. & Pineda, Salvador, 2021. "An exact dynamic programming approach to segmented isotonic regression," Omega, Elsevier, vol. 105(C).
    14. Lutz Bornmann & Rüdiger Mutz, 2015. "Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2215-2222, November.
    15. Tan, Xiujie & Dong, Hanmin & Liu, Yishuang & Su, Xin & Li, Zixian, 2022. "Green bonds and corporate performance: A potential way to achieve green recovery," Renewable Energy, Elsevier, vol. 200(C), pages 59-68.
    16. Siu-Tong Au & Rong Duan & Siamak Hesar & Wei Jiang, 2010. "A framework of irregularity enlightenment for data pre-processing in data mining," Annals of Operations Research, Springer, vol. 174(1), pages 47-66, February.
    17. Luca Lalor & Anatoliy Swishchuk, 2024. "Reinforcement Learning in Non-Markov Market-Making," Papers 2410.14504, arXiv.org, revised Nov 2024.
    18. Matúš Maciak & Ivan Mizera, 2016. "Regularization techniques in joinpoint regression," Statistical Papers, Springer, vol. 57(4), pages 939-955, December.
    19. Antoinette Deborah Martin & Ezat Ahmadzadeh & Inkyu Moon, 2022. "Privacy-Preserving Image Captioning with Deep Learning and Double Random Phase Encoding," Mathematics, MDPI, vol. 10(16), pages 1-14, August.
    20. Rodrigo R. Soares & Rudi Rocha & Michel Szklo, 2021. "American Delusion: Life Expectancy and Welfare in the US from an International Perspective," Working Papers 13, Instituto de Estudos para Políticas de Saúde.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:16:p:3019-:d:894277. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.