9 June 2021 in IEEE Access
Generating low-rank approximations of kernel matrices that arise in nonlinear machine learning techniques holds the potential to significantly alleviate the memory and computational burdens. A compelling approach centers on finding a concise set of exemplars or landmarks to reduce the number of similarity measure evaluations from quadratic to linear concerning the data size. However, a key challenge is to regulate tradeoffs between the quality of landmarks and resource consumption. Despite the volume of research in this area, current understanding is limited regarding the performance of landmark selection techniques in the presence of class-imbalanced data sets that are becoming increasingly prevalent in many applications. Hence, this paper provides a comprehensive empirical investigation using several real-world imbalanced data sets, including scientific data, by evaluating the quality of approximate low-rank decompositions and examining their influence on the accuracy of downstream tasks. Furthermore, we present a new landmark selection technique called Distance-based Importance Sampling and Clustering (DISC), in which the relative importance scores are computed for improving accuracy-efficiency tradeoffs compared to existing works that range from probabilistic sampling to clustering methods. The proposed landmark selection method follows a coarse-to-fine strategy to capture the intrinsic structure of complex data sets, allowing us to substantially reduce the computational complexity and memory footprint with minimal loss in accuracy.
26 December 2024 in Expert Systems with Applications
11 April 2024 in IEEE Access
MHMohammad Amin Hariri-Ardebili
Mohammad Amin Hariri-Ardebili
Machine learning often lacks transparent performance indicators, especially in generating point predictions. This paper addresses this limitation through conformal prediction, a non-parametric forecasting technique seamlessly integrating with regression algorithms to produce prediction intervals at specified confidence levels. A crucial element in conformal prediction is the non-conformity score, traditionally based on absolute residual errors. In this work, we propose a novel approach, introducing data-dependent weights for computing non-conformity scores. This enhancement, considering the distances of training instances from the test sample, aims to improve overall algorithm performance. Empirical investigations across various real-world regression data sets, including scientific data, evaluate the efficiency and validity of prediction intervals from different uncertainty quantification methods. Results show that prediction intervals computed with data-dependent weights adapt to estimator uncertainty, offering more precise predictions in certain scenarios and appropriately conservative predictions in high uncertainty situations. Additionally, we compare predictive regions generated by conformal prediction with those from Gaussian Process Regression (GPR) for scientific data in structural engineering. To augment conformal prediction, we explore Conformalized Quantile Regression (CQR), a recent innovation combining conformal prediction with classical quantile regression, claiming full adaptability to heteroscedasticity. Our findings indicate that conformal prediction methods using data-dependent non-conformity scores achieve a 1% higher effective coverage level and a 15% reduction in prediction interval widths compared to other methods. The comparative analysis against GPR and CQR underscores the practical value of our approach in providing accurate prediction intervals in scientific and engineering domains.