IDEAS home Printed from https://ideas.repec.org/a/wsi/ijitdm/v10y2011i01ns0219622011004282.html
   My bibliography  Save this article

Ensemble Of Software Defect Predictors: An Ahp-Based Evaluation Method

Author

Listed:
  • YI PENG

    (School of Management and Economics, University of Electronic Science and Technology of China, Chengdu, P. R. China, 610054, P. R. China)

  • GANG KOU

    (School of Management and Economics, University of Electronic Science and Technology of China, Chengdu, P. R. China, 610054, P. R. China)

  • GUOXUN WANG

    (School of Management and Economics, University of Electronic Science and Technology of China, Chengdu, P. R. China, 610054, P. R. China)

  • WENSHUAI WU

    (School of Management and Economics, University of Electronic Science and Technology of China, Chengdu, P. R. China, 610054, P. R. China)

  • YONG SHI

    (College of Information Science & Technology, University of Nebraska at Omaha, Omaha, NE 68182, USA;
    CAS Research Center on Fictitious Economy and Data Sciences, Beijing 100080, China)

Abstract

Classification algorithms that help to identify software defects or faults play a crucial role in software risk management. Experimental results have shown that ensemble of classifiers are often more accurate and robust to the effects of noisy data, and achieve lower average error rate than any of the constituent classifiers. However, inconsistencies exist in different studies and the performances of learning algorithms may vary using different performance measures and under different circumstances. Therefore, more research is needed to evaluate the performance of ensemble algorithms in software defect prediction. The goal of this paper is to assess the quality of ensemble methods in software defect prediction with the analytic hierarchy process (AHP), which is a multicriteria decision-making approach that prioritizes decision alternatives based on pairwise comparisons. Through the application of the AHP, this study compares experimentally the performance of several popular ensemble methods using 13 different performance metrics over 10 public-domain software defect datasets from the NASA Metrics Data Program (MDP) repository. The results indicate that ensemble methods can improve the classification results of software defect prediction in general and AdaBoost gives the best results. In addition, tree and rule based classifiers perform better in software defect prediction than other types of classifiers included in the experiment. In terms of single classifier, K-nearest-neighbor, C4.5, and Naïve Bayes tree ranked higher than other classifiers.

Suggested Citation

  • Yi Peng & Gang Kou & Guoxun Wang & Wenshuai Wu & Yong Shi, 2011. "Ensemble Of Software Defect Predictors: An Ahp-Based Evaluation Method," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 10(01), pages 187-206.
  • Handle: RePEc:wsi:ijitdm:v:10:y:2011:i:01:n:s0219622011004282
    DOI: 10.1142/S0219622011004282
    as

    Download full text from publisher

    File URL: http://www.worldscientific.com/doi/abs/10.1142/S0219622011004282
    Download Restriction: Access to full text is restricted to subscribers

    File URL: https://libkey.io/10.1142/S0219622011004282?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Binh Thai Pham & Dieu Tien Bui & Indra Prakash & M. B. Dholakia, 2016. "Rotation forest fuzzy rule-based classifier ensemble for spatial prediction of landslides using GIS," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 83(1), pages 97-127, August.
    2. Yi Peng, 2015. "Regional earthquake vulnerability assessment using a combination of MCDM methods," Annals of Operations Research, Springer, vol. 234(1), pages 95-110, November.
    3. Jung-Yu Lai & Juite Wang & Yi-Hsuan Chiu, 2021. "Evaluating blockchain technology for reducing supply chain risks," Information Systems and e-Business Management, Springer, vol. 19(4), pages 1089-1111, December.
    4. Changsheng Lin & Gang Kou & Daji Ergu, 2013. "An improved statistical approach for consistency test in AHP," Annals of Operations Research, Springer, vol. 211(1), pages 289-299, December.
    5. Kevin Kam Fung Yuen, 2014. "The Least Penalty Optimization Prioritization Operators for the Analytic Hierarchy Process: A Revised Case of Medical Decision Problem of Organ Transplantation," Systems Engineering, John Wiley & Sons, vol. 17(4), pages 442-461, December.
    6. Wenshuai Wu & Gang Kou, 2016. "A group consensus model for evaluating real estate investment alternatives," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 2(1), pages 1-10, December.
    7. Peng, Yi & Kou, Gang & Wang, Guoxun & Shi, Yong, 2011. "FAMCDM: A fusion approach of MCDM methods to rank multiclass classification algorithms," Omega, Elsevier, vol. 39(6), pages 677-689, December.
    8. Gang Kou & Wenshuai Wu, 2014. "Multi-criteria decision analysis for emergency medical service assessment," Annals of Operations Research, Springer, vol. 223(1), pages 239-254, December.
    9. Daji Ergu & Gang Kou & János Fülöp & Yong Shi, 2014. "Further Discussions on Induced Bias Matrix Model for the Pair-Wise Comparison Matrix," Journal of Optimization Theory and Applications, Springer, vol. 161(3), pages 980-993, June.
    10. Yang, Chih-Hao & Lee, Kuen-Chang, 2020. "Developing a strategy map for forensic accounting with fraud risk management: An integrated balanced scorecard-based decision model," Evaluation and Program Planning, Elsevier, vol. 80(C).
    11. Thierno M. L. Diallo & Sébastien Henry & Yacine Ouzrout & Abdelaziz Bouras, 2018. "Data-Based Fault Diagnosis Model Using a Bayesian Causal Analysis Framework," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 17(02), pages 583-620, March.
    12. Kou, Gang & Lin, Changsheng, 2014. "A cosine maximization method for the priority vector derivation in AHP," European Journal of Operational Research, Elsevier, vol. 235(1), pages 225-232.
    13. Daji Ergu & Gang Kou, 2012. "Questionnaire design improvement and missing item scores estimation for rapid and efficient decision making," Annals of Operations Research, Springer, vol. 197(1), pages 5-23, August.
    14. Wenshuai Wu & Yi Peng, 2016. "Extension of grey relational analysis for facilitating group consensus to oil spill emergency management," Annals of Operations Research, Springer, vol. 238(1), pages 615-635, March.
    15. Wenshuai Wu & Yi Peng, 2016. "Extension of grey relational analysis for facilitating group consensus to oil spill emergency management," Annals of Operations Research, Springer, vol. 238(1), pages 615-635, March.
    16. Ergu, Daji & Kou, Gang & Peng, Yi & Shi, Yong, 2011. "A simple method to improve the consistency ratio of the pair-wise comparison matrix in ANP," European Journal of Operational Research, Elsevier, vol. 213(1), pages 246-259, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wsi:ijitdm:v:10:y:2011:i:01:n:s0219622011004282. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tai Tone Lim (email available below). General contact details of provider: http://www.worldscinet.com/ijitdm/ijitdm.shtml .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.