Evolution of Software Development Effort and Cost Estimation Techniques: Five Decades Study Using Automated Text Mining Approach

Software development effort and cost estimation (SDECE) is one of the most important tasks in the field of software engineering. A large number of research papers have been published on this topic in the last five decades. Investigating research trends using a systematic literature review when such a large number of research papers are published is a very tedious and time-consuming task. Therefore, in this research paper, we propose a generic automated text mining framework to investigate research trends by analyzing the title, author’s keywords, and abstract of the research papers. The proposed framework is used to investigate research trends by analyzing the title, keywords, and abstract of select 1015 research papers published on SDECE in the last five decades. We have identified the most popular SDECE techniques in each decade to understand how SDECE has evolved in the past five decades. It is found that artificial neural network, fuzzy logic, regression, analogy-based approach, and COCOMO methods are the most used techniques for SDECE followed by optimization, use case point, machine learning, and function point analysis. The NASA and ISBSG are the most used dataset for SDECE. The MMRE, MRE, and PRED are the most used accuracy measures for SDECE. Results of the proposed framework are validated by comparing it with the outcome of the previously published review work and we found that the results are consistent. We have also carried out a detailed bibliometric analysis and metareview of the review and survey papers published on SDECE. This research study is significant for the development of new models for cost and effort estimations.