TY - JOUR
T1 - A classification scheme to improve conclusion instability using Bellwether moving windows
AU - Mensah, Solomon
AU - Kudjo, Patrick Kwaku
N1 - Publisher Copyright:
© 2022 John Wiley & Sons Ltd.
PY - 2022/9
Y1 - 2022/9
N2 - Context: The use of a subset of recently completed and exemplary data, namely, Bellwether moving window (BMW) has proven successful to result in improved accuracy in software effort estimation (SEE). These outcomes were achieved based on the theory that estimation outcome of a future event depends on previous events. Thus, the existence of a BMW yield improved prediction accuracy for new project estimation. However, the conclusion instability problem across learners still threatens the reliability of SEE for new projects. Such instability concerns are attributed to the data subset considered for the training and validation needs of learners. Objective: To investigate whether the use of BMWs together with an effort classification scheme can minimize the conclusion instability problem across learners. Method: We apply a Bellwether method comprising of three operators, namely, SORT+CLUSTER, GENERATE_TPM, and APPLY to sample the BMW from a pool of chronological projects from the Maxwell and International Software Benchmarking Standards Group (ISBSG) datasets. The sampled BMW is benchmarked against the entire collection of preprocessed projects, namely, growing portfolio to evaluate prediction and classification accuracy across a set of learners–ElasticNet regression, deep neural networks, and automatically transformed linear model. Results: (1) BMW exists in the studied projects and (2) training the learners with a BMW of average window size 28.5%–75.5% of the growing portfolio (not older than 3 years) relatively minimizes the conclusion instability of prediction results. Conclusion: When BMWs are available, we recommend their use for estimating the effort for a new project to minimize the conclusion instability problem.
AB - Context: The use of a subset of recently completed and exemplary data, namely, Bellwether moving window (BMW) has proven successful to result in improved accuracy in software effort estimation (SEE). These outcomes were achieved based on the theory that estimation outcome of a future event depends on previous events. Thus, the existence of a BMW yield improved prediction accuracy for new project estimation. However, the conclusion instability problem across learners still threatens the reliability of SEE for new projects. Such instability concerns are attributed to the data subset considered for the training and validation needs of learners. Objective: To investigate whether the use of BMWs together with an effort classification scheme can minimize the conclusion instability problem across learners. Method: We apply a Bellwether method comprising of three operators, namely, SORT+CLUSTER, GENERATE_TPM, and APPLY to sample the BMW from a pool of chronological projects from the Maxwell and International Software Benchmarking Standards Group (ISBSG) datasets. The sampled BMW is benchmarked against the entire collection of preprocessed projects, namely, growing portfolio to evaluate prediction and classification accuracy across a set of learners–ElasticNet regression, deep neural networks, and automatically transformed linear model. Results: (1) BMW exists in the studied projects and (2) training the learners with a BMW of average window size 28.5%–75.5% of the growing portfolio (not older than 3 years) relatively minimizes the conclusion instability of prediction results. Conclusion: When BMWs are available, we recommend their use for estimating the effort for a new project to minimize the conclusion instability problem.
KW - Bellwether moving window
KW - Eubank's optimal spacing theory
KW - conclusion instability
KW - growing portfolio
KW - software effort classification
UR - http://www.scopus.com/inward/record.url?scp=85134480015&partnerID=8YFLogxK
U2 - 10.1002/smr.2488
DO - 10.1002/smr.2488
M3 - Article
AN - SCOPUS:85134480015
SN - 2047-7481
VL - 34
JO - Journal of software: Evolution and Process
JF - Journal of software: Evolution and Process
IS - 9
M1 - e2488
ER -