The performance of software fault localization techniques is critical to software debugging and the reliability of software. Spectrum-based fault localization (SBFL) and mutation-based fault localization (MBFL) are the two most popular fault localization methods. However, the accuracies of the two methods are still limited. For example, only 10.63% of faults can be detected by inspecting the top 3 suspicious elements reported by Ochiai, which is a famous SBFL technique. Unfortunately, programmers only examine the first few suspicious elements before losing patience. Since the information used in SBFL and MBFL are quite different and complementary, this paper proposes a novel approach by combining spectrum and mutation to improve the fault localization accuracy. First, the faulty program is evaluated by using SBFL, and the potential faulty statements are ranked according to their suspiciousness. Then, mutants of the program are generated and executed by MBFL. Finally, the statements that are ranked in the top tied n by SBFL are evaluated and reranked according to their mutation scores. Experiments are carried on the Defects4J benchmark and the results reveal that the accuracy of the proposed approach outperforms those of the SBFL and MBFL techniques. In terms of the faults located by inspecting the top 1 suspicious elements, the SMFL techniques detect at least 2.36 times more faults than two SBFL techniques (DStar and Ochiai) and detect at least 1.86 times more faults than two MBFL techniques (MUSE and Metallaxis).

Security vulnerability prediction (SVP) can identify potential vulnerable modules in advance and then help developers to allocate most of the test resources to these modules. To evaluate the performance of different SVP methods, we should take the security audit and code inspection into account and then consider effort-aware performance measures (such as $ACC$ and $P_{\rm opt}$ ). However, to the best of our knowledge, the effectiveness of different SVP methods has not been thoroughly investigated in terms of effort-aware performance measures. In this article, we consider 48 different SVP methods, of which 36 are supervised methods and 12 are unsupervised methods. For the supervised methods, we consider 34 software-metric-based methods and two text-mining-based methods. For the software-metric-based methods, in addition to a large number of classification methods, we also consider four state-of-the-art methods (i.e., EALR, OneWay, CBS, and MULTI) proposed in recent effort-aware just-in-time defect prediction studies. For text-mining-based methods, we consider the Bag-of-Word model and the term-frequency-inverse-document-frequency model. For the unsupervised methods, all the modules are ranked in the ascendent order based on a specific metric. Since 12 software metrics are considered when measuring extracted modules, there are 12 different unsupervised methods. To the best of our knowledge, over 40 SVP methods have not been considered in previous SVP studies. In our large-scale empirical studies, we use three real open-source web applications written in PHP as benchmark. These three web applications include 3466 modules and 223 vulnerabilities in total. We evaluate these SVP methods both in the within-project SVP scenario and the cross-project SVP scenario. Empirical results show that two unsupervised methods [i.e., lines of code (LOC) and Halstead’s volume (HV)] and four recently proposed state-of-the-art supervised methods (i.e., MULTI, OneWay, CBS, and EALR) can achieve better performance than the other methods in terms of effort-aware performance measures. Then, we analyze the reasons why these six methods can achieve better performance. For example, when using 20% of the entire efforts, we find that these six methods always require more modules to be inspected, especially for unsupervised methods LOC and HV. Finally, from the view of practical vulnerability localization, we find that all the unsupervised methods and the OneWay method have high false alarms before finding the first vulnerable module. This may have an impact on developers’ confidence and tolerance, and supervised methods (especially MULTI and text-mining-based methods) are preferred.

Cross-project defect prediction (CPDP) methods can be used when the target project is a new project or lacks enough labeled program modules. In these new target projects, we can easily extract and then measure these modules with software measurement tools. However, labeling these program modules is time-consuming, error-prone and requires professional domain knowledge. Moreover, directly using labeled modules in the other projects (i.e., the source projects) can not achieve satisfactory performance due to the large data distribution difference in most cases. In this article, to our best knowledge, we are the first to propose a novel method ALTRA, which can utilize both active learning and TrAdaBoost to alleviate this issue. In particular, we firstly use Burak filter to select similar labeled modules from the source project after analyzing the unlabeled modules in the target project. Then we use active learning to choose representative unlabeled modules from the target project and ask experts to label the type (i.e., defective or non-defective) of these modules. Later, we use TrAdaBoost to determine the weights of labeled modules in the source project and the target project, and then construct the model via weighted support vector machine. After selecting a small number of modules (i.e., only 5% modules ) in the target project, we terminate the method ALTRA and return the final constructed model. To show the effectiveness of our proposed method ALTRA, we choose 10 large-scale open-source projects from different application domains. In terms of both F1 and AUC performance indicators, we find ALTRA can perform significantly better than seven state-of-the-art CPDP baselines. Moreover, we also show that the usage of Burak filter, the uncertainty active learning strategy, the class imbalanced learning method and TrAdaBoost are competitive in our proposed method ALTRA.

Software defect prediction can identify potential defective modules in advance and then provide guidances for software testers to allocate more testing resources on these modules. During the gathering process for defect prediction datasets, if multiple metrics are used to measure the program modules, it will result in curse of dimensionality. Feature selection is one of effective methods to alleviate this problem. However, designing effective feature selection methods is a great challenge. Motivated by the idea of search based software engineering, we formalize this problem as a multi-objective optimization problem, and then propose novel method MOFES. To verify the effectiveness of our proposed method, we choose PROMISE dataset gathered from real projects, and compare MOFES with some classical baseline methods. Final results show that our method has the advantages of selecting less features and achieving better prediction performance in most projects while its computational cost is acceptable.