Techniques |
Applicability |
TCP
|
Industry Motivation
|
Experiment subject(s) |
Industrial Partner |
Programming Language |
Open-source C and Java projects (up to 670 TCs) + synthetic data for scalability measure (up to 1M TCs)
Open-source, medium scale
Synthetic data, very large scale |
|
C, Java |
Effectiveness Metrics |
Efficiency Metrics |
Other Metrics |
Average Percentage of Faults Detected (APFD)
|
Execution time, Total/End-to-end time, Scalability
|
|
Information Approach |
Algorithm Approach |
Open Challenges |
Test code
|
Similarity / distance-based
|
Adapt algorithm ti include other objective functions in addition to dissimilarity.
|
Abstract
Many test case prioritization criteria have been proposed for speeding up fault detection. Among them, similarity-based approaches give priority to the test cases that are the most dissimilar from those already selected. However, the proposed criteria do not scale up to handle the many thousands or even some millions test suite sizes of modern industrial systems and simple heuristics are used instead. We introduce the FAST family of test case prioritization techniques that radically changes this landscape by borrowing algorithms commonly exploited in the big data domain to find similar items. FAST techniques provide scalable similarity-based test case prioritization in both white-box and black-box fashion. The results from experimentation on real world C and Java subjects show that the fastest members of the family outperform other black-box approaches in e ciency with no significant impact on e ectiveness, and also outperform white-box approaches, including greedy ones, if preparation time is not counted. A simulation study of scalability shows that one FAST technique can prioritize a million test cases in less than 20 minutes.