Prescriptive analytics extends on predictive analytics by allowing to estimate an outcome in function of control variables, allowing as such to establish the required level of control variables for realizing a desired outcome. Uplift modeling is at the heart of prescriptive analytics and aims at estimating the net difference in an outcome resulting from a specific action or treatment that is applied. In this article, a structured and detailed literature survey on uplift modeling is provided by identifying and contrasting various groups of approaches. In addition, evaluation metrics for assessing the performance of uplift models are reviewed. An experimental evaluation on four real-world data sets provides further insight into their use. Uplift random forests are found to be consistently among the best performing techniques in terms of the Qini and Gini measures, although considerable variability in performance across the various data sets of the experiments is observed. In addition, uplift models are frequently observed to be unstable and display a strong variability in terms of performance across different folds in the cross-validation experimental setup. This potentially threatens their actual use for business applications. Moreover, it is found that the available evaluation metrics do not provide an intuitively understandable indication of the actual use and performance of a model. Specifically, existing evaluation metrics do not facilitate a comparison of uplift models and predictive models and evaluate performance either at an arbitrary cutoff or over the full spectrum of potential cutoffs. In conclusion, we highlight the instability of uplift models and the need for an application-oriented approach to assess uplift models as prime topics for further research.
7 December 2020 in Journal of the Operational Research Society
Application scorecards allow to assess the creditworthiness of loan applicants and decide on acceptance. The accuracy of scorecards is of crucial importance for minimizing bad debt loss and maximizing returns. In this paper, we extend upon prior benchmarking studies that experimentally compare the performance of classification techniques to discriminate between good and bad applications. We evaluate a range of cost-sensitive learning methods in terms of their ability to boost the profitability of scorecards. These methods allow to take into account the variable misclassification costs that are involved in rejecting good loan applications and accepting bad loan applications. An approach is proposed to estimate these misclassification costs, and various approaches to handle missing credit bureau scores are evaluated. The results of a case study involving a Romanian nonbanking financial institution (NBFI) indicate that cost-sensitive learning complements the existing state-of-the-art scorecard of the NBFI. The best performing cost-sensitive models are found to increase profitability across the three business channels, with a single-digit improvement for two of the channels and a double-digit increase for the other one. The result is partly explained by the default rate, which is higher for this latter channel and therefore offers greater potential for improving profitability.
The effectiveness of machine learning in evaluating the creditworthiness of loan applicants has been demonstrated for a long time. However, there is concern that the use of automated decision-making processes may result in unequal treatment of groups or individuals, potentially leading to discriminatory outcomes. This paper seeks to address this issue by evaluating the effectiveness of 12 leading bias mitigation methods across 5 different fairness metrics, as well as assessing their accuracy and potential profitability for financial institutions. Through our analysis, we have identified the challenges associated with achieving fairness while maintaining accuracy and profitabiliy, and have highlighted both the most successful and least successful mitigation methods. Ultimately, our research serves to bridge the gap between experimental machine learning and its practical applications in the finance industry.
This paper presents a sentiment analysis solution on tweets using Recurrent Neural Networks (RNNs). The method is can classifying tweets with an 80.74% accuracy rate, considering a binary task, after experimenting with 20 different design approaches. The solution integrates an attention mechanism aiming to enhance the network, with a two-way localization system: at memory cell level and at network level. We present an in-depth literature review for Twitter sentiment analysis and the building blocks that grounded the design decisions of our solution, employed as a core classification component within a sentiment indicator of the SynergyCrowds platform.
This article presents a tailored majority voting approach for enhancing the consistency and reliability of sentiment analysis in online product reviews. The methodology addresses discrepancies in sentiment classification by leveraging sentiment labels from multiple automated tools and implementing a robust majority decision rule. This consensus-based approach significantly enhances the trustworthiness and consistency of sentiment analysis outcomes, serving as a dependable foundation for training more precise sentiment analysis models. The data labeled with our method was utilized to train deep learning models, achieving competitive accuracy with significantly less data. The findings demonstrate the effectiveness of the method in producing results comparable to commercial tools while ensuring data consistency for model training.