With the advancements in processing units and easy availability of cloud-based GPU servers, many deep learning-based methods have been proposed for Aspect Level Sentiment Classification (ALSC) literature. With this increase in the number of deep learning methods proposed in ALSC literature, it has become difficult to ascertain the performance difference of one method over the other. To this end, our study provides a statistical comparison of the performance of 35 recent deep learning methods with respect to three performance metrics-Accuracy, Macro F1 score, and Time. The methods are evaluated for eight benchmark datasets. In this study, the statistical comparison is based on Friedman, Nemenyi, and Wilcoxon tests. As per the results of statistical tests, the top-ranking methods could not significantly outperform several other methods in terms of Accuracy and Macro F1 score and performed poorly on-time metric. However, the time taken by any method is crucial to analyze the overall performance. Thus, this study aids the selection of the Deep Learning method, which maximizes the accuracy and Macro F1 score and takes minimal time. Our study also establishes a framework for validating the performance of new and alternate methods in ALSC that can be helpful for researchers and practitioners working in this area.
Machine learning techniques have been earnestly explored by many software engineering researchers. At present state of art, there is no conclusive evidence on the kind of machine learning techniques which are most accurate and efficient for software defect prediction but some recent studies suggest that combining multiple machine learners, that is, ensemble learning, may be a more accurate alternative. This study contributes to software defect prediction literature by systematically evaluating the predictive accuracy of three well known homogeneous ensemble methods - Bagging, Boosting, and Rotation Forest, utilizing fifteen important underlying base learners, by exploiting the data of nine open source object-oriented systems obtained from the PROMISE repository. Results indicate while Bagging and Boosting may result in AUC performance loss, AUC performance improvement results in twelve of the fifteen investigated base learners with Rotation Forest ensemble.