IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i13p2219-d847282.html
   My bibliography  Save this article

Permutation Tests for Metaheuristic Algorithms

Author

Listed:
  • Mahamed G. H. Omran

    (Centre for Applied Mathematics and Bioinformatics, and Computer Science Department, Gulf University for Science & Technology, Hawally 32093, Kuwait)

  • Maurice Clerc

    (Independent Consultant, France)

  • Fatme Ghaddar

    (Computer Science Department, Gulf University for Science & Technology, Hawally 32093, Kuwait)

  • Ahmad Aldabagh

    (Computer Science Department, Gulf University for Science & Technology, Hawally 32093, Kuwait)

  • Omar Tawfik

    (Computer Science Department, Gulf University for Science & Technology, Hawally 32093, Kuwait)

Abstract

Many metaheuristic approaches are inherently stochastic. In order to compare such methods, statistical tests are needed. However, choosing an appropriate test is not trivial, given that each test has some assumptions about the distribution of the underlying data that must be true before it can be used. Permutation tests (P-Tests) are statistical tests with minimal number of assumptions. These tests are simple, intuitive and nonparametric. In this paper, we argue researchers in the field of metaheuristics to adopt P-Tests to compare their algorithms. We define two statistic tests and then present an algorithm that uses them to compute the p -value. The proposed process is used to compare 5 metaheuristic algorithms on 10 benchmark functions. The resulting p -values are compared with the p -values of two widely used statistical tests. The results show that the proposed P-test is generally consistent with the classical tests, but more conservative in few cases.

Suggested Citation

  • Mahamed G. H. Omran & Maurice Clerc & Fatme Ghaddar & Ahmad Aldabagh & Omar Tawfik, 2022. "Permutation Tests for Metaheuristic Algorithms," Mathematics, MDPI, vol. 10(13), pages 1-15, June.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:13:p:2219-:d:847282
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/13/2219/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/13/2219/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chou, Jui-Sheng & Truong, Dinh-Nhat, 2021. "A novel metaheuristic optimizer inspired by behavior of jellyfish in ocean," Applied Mathematics and Computation, Elsevier, vol. 389(C).
    2. Hashim, Fatma A. & Houssein, Essam H. & Hussain, Kashif & Mabrouk, Mai S. & Al-Atabany, Walid, 2022. "Honey Badger Algorithm: New metaheuristic algorithm for solving optimization problems," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 192(C), pages 84-110.
    3. Aickin, M. & Gensler, H., 1996. "Adjusting for multiple testing when reporting research results: The Bonferroni vs Holm methods," American Journal of Public Health, American Public Health Association, vol. 86(5), pages 726-728.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Salil Madhav Dubey & Hari Mohan Dubey & Surender Reddy Salkuti, 2022. "Modified Quasi-Opposition-Based Grey Wolf Optimization for Mathematical and Electrical Benchmark Problems," Energies, MDPI, vol. 15(15), pages 1-29, August.
    2. Nikola Ivković & Robert Kudelić & Matej Črepinšek, 2022. "Probability and Certainty in the Performance of Evolutionary and Swarm Optimization Algorithms," Mathematics, MDPI, vol. 10(22), pages 1-25, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Araby Mahdy & Abdullah Shaheen & Ragab El-Sehiemy & Ahmed Ginidi & Saad F. Al-Gahtani, 2023. "Single- and Multi-Objective Optimization Frameworks of Shape Design of Tubular Linear Synchronous Motor," Energies, MDPI, vol. 16(5), pages 1-27, March.
    2. Liqiong Huang & Yuanyuan Wang & Yuxuan Guo & Gang Hu, 2022. "An Improved Reptile Search Algorithm Based on Lévy Flight and Interactive Crossover Strategy to Engineering Application," Mathematics, MDPI, vol. 10(13), pages 1-39, July.
    3. Örnek, Bülent Nafi & Aydemir, Salih Berkan & Düzenli, Timur & Özak, Bilal, 2022. "A novel version of slime mould algorithm for global optimization and real world engineering problems," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 198(C), pages 253-288.
    4. Hyojung Tak & Gregory Ruhnke & Ya-Chen Shih, 2015. "The Association between Patient-Centered Attributes of Care and Patient Satisfaction," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 8(2), pages 187-197, April.
    5. Essam H. Houssein & Awny Sayed, 2023. "Dynamic Candidate Solution Boosted Beluga Whale Optimization Algorithm for Biomedical Classification," Mathematics, MDPI, vol. 11(3), pages 1-27, January.
    6. Zhang, Ziyuan & Wang, Jianzhou & Wei, Danxiang & Luo, Tianrui & Xia, Yurui, 2023. "A novel ensemble system for short-term wind speed forecasting based on Two-stage Attention-Based Recurrent Neural Network," Renewable Energy, Elsevier, vol. 204(C), pages 11-23.
    7. Jian Zhao & Bochen Zhang & Xiwang Guo & Liang Qi & Zhiwu Li, 2022. "Self-Adapting Spherical Search Algorithm with Differential Evolution for Global Optimization," Mathematics, MDPI, vol. 10(23), pages 1-31, November.
    8. Hegazy Rezk & A. G. Olabi & Mohammad Ali Abdelkareem & Abdul Hai Alami & Enas Taha Sayed, 2023. "Optimal Parameter Determination of Membrane Bioreactor to Boost Biohydrogen Production-Based Integration of ANFIS Modeling and Honey Badger Algorithm," Sustainability, MDPI, vol. 15(2), pages 1-13, January.
    9. Ahmed Ginidi & Sherif M. Ghoneim & Abdallah Elsayed & Ragab El-Sehiemy & Abdullah Shaheen & Attia El-Fergany, 2021. "Gorilla Troops Optimizer for Electrically Based Single and Double-Diode Models of Solar Photovoltaic Systems," Sustainability, MDPI, vol. 13(16), pages 1-28, August.
    10. Fatmah Y. Assiri & Mahmoud Ragab, 2023. "Optimal Deep-Learning-Based Cyberattack Detection in a Blockchain-Assisted IoT Environment," Mathematics, MDPI, vol. 11(19), pages 1-16, September.
    11. Colette Cunningham-Myrie & Katherine P Theall & Novie Younger-Coleman & Lisa-Gaye Greene & Parris Lyew-Ayee & Rainford Wilks, 2021. "Associations of neighborhood physical and crime environments with obesity-related outcomes in Jamaica," PLOS ONE, Public Library of Science, vol. 16(4), pages 1-19, April.
    12. Ghareeb Moustafa & Ali M. El-Rifaie & Idris H. Smaili & Ahmed Ginidi & Abdullah M. Shaheen & Ahmed F. Youssef & Mohamed A. Tolba, 2023. "An Enhanced Dwarf Mongoose Optimization Algorithm for Solving Engineering Problems," Mathematics, MDPI, vol. 11(15), pages 1-26, July.
    13. Eleonora Cataldo & Maddalena Fucile & Giovan Battista Mattii, 2022. "Effects of Kaolin and Shading Net on the Ecophysiology and Berry Composition of Sauvignon Blanc Grapevines," Agriculture, MDPI, vol. 12(4), pages 1-21, March.
    14. Eslami, N. & Yazdani, S. & Mirzaei, M. & Hadavandi, E., 2022. "Aphid–Ant Mutualism: A novel nature-inspired​ metaheuristic algorithm for solving optimization problems," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 201(C), pages 362-395.
    15. Van de Velde, Liesbeth & Verbeke, Wim & Popp, Michael & Van Huylenbroeck, Guido, 2010. "The importance of message framing for providing information about sustainability and environmental aspects of energy," Energy Policy, Elsevier, vol. 38(10), pages 5541-5549, October.
    16. Mohamed Abdel-Basset & Reda Mohamed & Ripon K. Chakrabortty & Michael J. Ryan & Attia El-Fergany, 2021. "An Improved Artificial Jellyfish Search Optimizer for Parameter Identification of Photovoltaic Models," Energies, MDPI, vol. 14(7), pages 1-33, March.
    17. Muhyaddin Rawa & Abdullah Abusorrah & Yusuf Al-Turki & Martin Calasan & Mihailo Micev & Ziad M. Ali & Saad Mekhilef & Hussain Bassi & Hatem Sindi & Shady H. E. Abdel Aleem, 2022. "Estimation of Parameters of Different Equivalent Circuit Models of Solar Cells and Various Photovoltaic Modules Using Hybrid Variants of Honey Badger Algorithm and Artificial Gorilla Troops Optimizer," Mathematics, MDPI, vol. 10(7), pages 1-31, March.
    18. Muhammad Haris Khan & Abasin Ulasyar & Abraiz Khattak & Haris Sheh Zad & Mohammad Alsharef & Ahmad Aziz Alahmadi & Nasim Ullah, 2022. "Optimal Sizing and Allocation of Distributed Generation in the Radial Power Distribution System Using Honey Badger Algorithm," Energies, MDPI, vol. 15(16), pages 1-18, August.
    19. Chenyang Gao & Teng Li & Yuelin Gao & Ziyu Zhang, 2024. "A Comprehensive Multi-Strategy Enhanced Biogeography-Based Optimization Algorithm for High-Dimensional Optimization and Engineering Design Problems," Mathematics, MDPI, vol. 12(3), pages 1-35, January.
    20. Natalia Nehrebecka, 2017. "Probability-of-default curve calibration and validation of internal rating systems," IFC Bulletins chapters, in: Bank for International Settlements (ed.), Statistical implications of the new financial landscape, volume 43, Bank for International Settlements.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:13:p:2219-:d:847282. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.