IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i22p4364-d978386.html
   My bibliography  Save this article

Probability and Certainty in the Performance of Evolutionary and Swarm Optimization Algorithms

Author

Listed:
  • Nikola Ivković

    (Faculty of Organization and Informatics, University of Zagreb, Pavlinska 2, 42000 Varaždin, Croatia)

  • Robert Kudelić

    (Faculty of Organization and Informatics, University of Zagreb, Pavlinska 2, 42000 Varaždin, Croatia)

  • Matej Črepinšek

    (Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, Slovenia)

Abstract

Reporting the empirical results of swarm and evolutionary computation algorithms is a challenging task with many possible difficulties. These difficulties stem from the stochastic nature of such algorithms, as well as their inability to guarantee an optimal solution in polynomial time. This research deals with measuring the performance of stochastic optimization algorithms, as well as the confidence intervals of the empirically obtained statistics. Traditionally, the arithmetic mean is used for measuring average performance, but we propose quantiles for measuring average, peak and bad-case performance, and give their interpretations in a relevant context for measuring the performance of the metaheuristics. In order to investigate the differences between arithmetic mean and quantiles, and to confirm possible benefits, we conducted experiments with 7 stochastic algorithms and 20 unconstrained continuous variable optimization problems. The experiments showed that median was a better measure of average performance than arithmetic mean, based on the observed solution quality. Out of 20 problem instances, a discrepancy between the arithmetic mean and median happened in 6 instances, out of which 5 were resolved in favor of median and 1 instance remained unresolved as a near tie. The arithmetic mean was completely inadequate for measuring average performance based on the observed number of function evaluations, while the 0.5 quantile (median) was suitable for that task. The quantiles also showed to be adequate for assessing peak performance and bad-case performance. In this paper, we also proposed a bootstrap method to calculate the confidence intervals of the probability of the empirically obtained quantiles. Considering the many advantages of using quantiles, including the ability to calculate probabilities of success in the case of multiple executions of the algorithm and the practically useful method of calculating confidence intervals, we recommend quantiles as the standard measure of peak, average and bad-case performance of stochastic optimization algorithms.

Suggested Citation

  • Nikola Ivković & Robert Kudelić & Matej Črepinšek, 2022. "Probability and Certainty in the Performance of Evolutionary and Swarm Optimization Algorithms," Mathematics, MDPI, vol. 10(22), pages 1-25, November.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:22:p:4364-:d:978386
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/22/4364/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/22/4364/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mahamed G. H. Omran & Maurice Clerc & Fatme Ghaddar & Ahmad Aldabagh & Omar Tawfik, 2022. "Permutation Tests for Metaheuristic Algorithms," Mathematics, MDPI, vol. 10(13), pages 1-15, June.
    2. Bolin Liao & Zhendai Huang & Xinwei Cao & Jianfeng Li, 2022. "Adopting Nonlinear Activated Beetle Antennae Search Algorithm for Fraud Detection of Public Trading Companies: A Computational Finance Approach," Mathematics, MDPI, vol. 10(13), pages 1-14, June.
    3. Qasim M. Zainel & Saad M. Darwish & Murad B. Khorsheed, 2022. "Employing Quantum Fruit Fly Optimization Algorithm for Solving Three-Dimensional Chaotic Equations," Mathematics, MDPI, vol. 10(21), pages 1-21, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Meichun Huang & Yunong Zhang, 2024. "Zhang Neuro-PID Control for Generalized Bi-Variable Function Projective Synchronization of Nonautonomous Nonlinear Systems with Various Perturbations," Mathematics, MDPI, vol. 12(17), pages 1-25, August.
    2. Haoran Zhao & Sen Guo, 2023. "Carbon Trading Price Prediction of Three Carbon Trading Markets in China Based on a Hybrid Model Combining CEEMDAN, SE, ISSA, and MKELM," Mathematics, MDPI, vol. 11(10), pages 1-21, May.
    3. Salil Madhav Dubey & Hari Mohan Dubey & Surender Reddy Salkuti, 2022. "Modified Quasi-Opposition-Based Grey Wolf Optimization for Mathematical and Electrical Benchmark Problems," Energies, MDPI, vol. 15(15), pages 1-29, August.
    4. Abdulaziz Almalaq & Tawfik Guesmi & Saleh Albadran, 2023. "A Hybrid Chaotic-Based Multiobjective Differential Evolution Technique for Economic Emission Dispatch Problem," Energies, MDPI, vol. 16(12), pages 1-34, June.
    5. Yuqi Fan & Sheng Zhang & Yaping Wang & Di Xu & Qisong Zhang, 2023. "An Improved Flow Direction Algorithm for Engineering Optimization Problems," Mathematics, MDPI, vol. 11(9), pages 1-31, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:22:p:4364-:d:978386. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.