IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i13p2288-d852593.html
   My bibliography  Save this article

Meta-Optimization of Dimension Adaptive Parameter Schema for Nelder–Mead Algorithm in High-Dimensional Problems

Author

Listed:
  • Žiga Rojec

    (Department of Electronics, Faculty of Electrical Engineering, University of Ljubljana, SI-1000 Ljubljana, Slovenia)

  • Tadej Tuma

    (Department of Electronics, Faculty of Electrical Engineering, University of Ljubljana, SI-1000 Ljubljana, Slovenia)

  • Jernej Olenšek

    (Department of Electronics, Faculty of Electrical Engineering, University of Ljubljana, SI-1000 Ljubljana, Slovenia)

  • Árpád Bűrmen

    (Department of Electronics, Faculty of Electrical Engineering, University of Ljubljana, SI-1000 Ljubljana, Slovenia)

  • Janez Puhan

    (Department of Electronics, Faculty of Electrical Engineering, University of Ljubljana, SI-1000 Ljubljana, Slovenia)

Abstract

Although proposed more than half a century ago, the Nelder–Mead simplex search algorithm is still widely used. Four numeric constants define the operations and behavior of the algorithm. The algorithm with the original constant values performs fine on most low-dimensional, but poorly on high-dimensional, problems. Therefore, to improve its behavior in high dimensions, several adaptive schemas setting the constants according to the problem dimension were proposed in the past. In this work, we present a novel adaptive schema obtained by a meta-optimization procedure. We describe a schema candidate with eight parameters subject to meta-optimization and define an objective function evaluating the candidate’s performance. The schema is optimized on up to 100-dimensional problems using the Parallel Simulated Annealing with Differential Evolution global method. The obtained global minimum represents the proposed schema. We compare the performance of the optimized schema with the existing adaptive schemas. The data profiles on the Gao–Han modified quadratic, Moré–Garbow–Hilstrom, and CUTEr (Constrained and Unconstrained Testing Environment, revisited) benchmark problem sets show that the obtained schema outperforms the existing adaptive schemas in terms of accuracy and convergence speed.

Suggested Citation

  • Žiga Rojec & Tadej Tuma & Jernej Olenšek & Árpád Bűrmen & Janez Puhan, 2022. "Meta-Optimization of Dimension Adaptive Parameter Schema for Nelder–Mead Algorithm in High-Dimensional Problems," Mathematics, MDPI, vol. 10(13), pages 1-16, June.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:13:p:2288-:d:852593
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/13/2288/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/13/2288/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. C.J. Price & I.D. Coope & D. Byatt, 2002. "A Convergent Variant of the Nelder–Mead Algorithm," Journal of Optimization Theory and Applications, Springer, vol. 113(1), pages 5-19, April.
    2. Fuchang Gao & Lixing Han, 2012. "Implementing the Nelder-Mead simplex algorithm with adaptive parameters," Computational Optimization and Applications, Springer, vol. 51(1), pages 259-277, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Árpád Bűrmen & Tadej Tuma, 2022. "Preface to the Special Issue on “Optimization Theory and Applications”," Mathematics, MDPI, vol. 10(24), pages 1-3, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Charles Audet & Christophe Tribes, 2018. "Mesh-based Nelder–Mead algorithm for inequality constrained optimization," Computational Optimization and Applications, Springer, vol. 71(2), pages 331-352, November.
    2. Ian Coope & Rachael Tappenden, 2019. "Efficient calculation of regular simplex gradients," Computational Optimization and Applications, Springer, vol. 72(3), pages 561-588, April.
    3. Riva-Palacio, Alan & Leisen, Fabrizio, 2021. "Compound vectors of subordinators and their associated positive Lévy copulas," Journal of Multivariate Analysis, Elsevier, vol. 183(C).
    4. Henri Bonnel & C. Yalçın Kaya, 2010. "Optimization Over the Efficient Set of Multi-objective Convex Optimal Control Problems," Journal of Optimization Theory and Applications, Springer, vol. 147(1), pages 93-112, October.
    5. Ivan Jericevich & Patrick Chang & Tim Gebbie, 2021. "Simulation and estimation of an agent-based market-model with a matching engine," Papers 2108.07806, arXiv.org, revised Aug 2021.
    6. C.J. Price & I.D. Coope, 2003. "Frame-Based Ray Search Algorithms in Unconstrained Optimization," Journal of Optimization Theory and Applications, Springer, vol. 116(2), pages 359-377, February.
    7. Rocha Filho, T.M. & Moret, M.A. & Chow, C.C. & Phillips, J.C. & Cordeiro, A.J.A. & Scorza, F.A. & Almeida, A.-C.G. & Mendes, J.F.F., 2021. "A data-driven model for COVID-19 pandemic – Evolution of the attack rate and prognosis for Brazil," Chaos, Solitons & Fractals, Elsevier, vol. 152(C).
    8. Min Xi & Wenyu Sun & Yannan Chen & Hailin Sun, 2020. "A derivative-free algorithm for spherically constrained optimization," Journal of Global Optimization, Springer, vol. 76(4), pages 841-861, April.
    9. Kota Matsui & Wataru Kumagai & Takafumi Kanamori, 2017. "Parallel distributed block coordinate descent methods based on pairwise comparison oracle," Journal of Global Optimization, Springer, vol. 69(1), pages 1-21, September.
    10. Chang, Kuo-Hao, 2015. "A direct search method for unconstrained quantile-based simulation optimization," European Journal of Operational Research, Elsevier, vol. 246(2), pages 487-495.
    11. Ronan Keane & H. Oliver Gao, 2021. "Fast Calibration of Car-Following Models to Trajectory Data Using the Adjoint Method," Transportation Science, INFORMS, vol. 55(3), pages 592-615, May.
    12. Michele Mininni & Giuseppe Orlando & Giovanni Taglialatela, 2021. "Challenges in approximating the Black and Scholes call formula with hyperbolic tangents," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 44(1), pages 73-100, June.
    13. Hsun-Heng Tsai & Chyun-Chau Fuh & Jeng-Rong Ho & Chih-Kuang Lin, 2021. "Design of Optimal Controllers for Unknown Dynamic Systems through the Nelder–Mead Simplex Method," Mathematics, MDPI, vol. 9(16), pages 1-14, August.
    14. Ralf Biehl, 2019. "Jscatter, a program for evaluation and analysis of experimental data," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-18, June.
    15. Mejía-de-Dios, Jesús-Adolfo & Mezura-Montes, Efrén & Toledo-Hernández, Porfirio, 2022. "Pseudo-feasible solutions in evolutionary bilevel optimization: Test problems and performance assessment," Applied Mathematics and Computation, Elsevier, vol. 412(C).
    16. Papo, David & Righetti, Marco & Fadiga, Luciano & Biscarini, Fabio & Zanin, Massimiliano, 2020. "A minimal model of hospital patients’ dynamics in COVID-19," Chaos, Solitons & Fractals, Elsevier, vol. 140(C).
    17. Pinto, Roberto, 2016. "Stock rationing under a profit satisficing objective," Omega, Elsevier, vol. 65(C), pages 55-68.
    18. Ingrida Steponavičė & Rob J. Hyndman & Kate Smith-Miles & Laura Villanova, 2017. "Dynamic algorithm selection for pareto optimal set approximation," Journal of Global Optimization, Springer, vol. 67(1), pages 263-282, January.
    19. Ferreiro, Ana M. & García-Rodríguez, José Antonio & Vázquez, Carlos & e Silva, E. Costa & Correia, A., 2019. "Parallel two-phase methods for global optimization on GPU," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 156(C), pages 67-90.
    20. He, Jie-Cao & Hsieh, Chang-Chieh & Huang, Zi-Wei & Lin, Shih-Kuei, 2023. "Valuation of callable range accrual linked to CMS Spread under generalized swap market model," International Review of Financial Analysis, Elsevier, vol. 90(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:13:p:2288-:d:852593. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.