Techniques |
Applicability |
TCP
|
Industry Motivation
Industry Evaluation
Industry Author
|
Experiment subject(s) |
Industrial Partner |
Programming Language |
Cisco videoconferencing system (460 TCs)
Industrial proprietary, small-scale |
Cisco (Norway) |
Unclear |
Effectiveness Metrics |
Efficiency Metrics |
Other Metrics |
Average Percentage of Faults Detected (APFD), Fault Detection Capability
|
|
|
Information Approach |
Algorithm Approach |
Open Challenges |
History-based, Fault-based
|
|
Utilize comprehensive cost-benefit measures, experiments in other industry domains
|
Abstract
Test prioritization is an effective technique used to reduce the amount of work required to support regression testing in continuous integration development. It aims at finding an optimal order of tests that can detect regressions faster, potentially increasing the frequency of software releases. Prioritization techniques based on test execution history use the results of preceding executions to determine an optimal order of regression tests in the succeeding test executions. In this paper, we investigate how can execution history be optimally used to increase the effectiveness of regression test prioritization. We analyze the effect of history time window on the fault detection effectiveness of prioritized regression tests. We report an experimental study using a data set from Cisco. The results suggest that varying the size of the window can considerably change the performance of regression testing. Our findings will potentially help developers and test teams in adjusting test prioritization techniques for achieving higher cost-effectiveness in continuous regression testing.