Anomaly detection on tabular data is widely used in fraud detection, predictive maintenance, and medical screening. While heterogeneous ensembles combining multiple detection paradigms achieve strong performance, their computational cost limits deployment in latency-sensitive or resource-constrained environments. We propose KD-AnomalyNet, a teacher–student framework that distills anomaly knowledge from a high-capacity ensemble into a lightweight neural model for efficient inference. Beyond performance replication, we study how anomaly representations transfer during distillation. To this end, we introduce a noise perturbation analysis that serves as a diagnostic probe for representation stability without introducing additional trainable components. Experiments on ten benchmark datasets show that the distilled model preserves up to 98.5% of the teacher’s AUC-ROC on the nine capacity-sufficient datasets (84.7% mean retention across all ten datasets) while achieving 26–181× inference speedups. Our analysis reveals which forms of anomaly knowledge transfer reliably—global outliers (78% transfer) and isolation-based detection (88% retention)—and which degrade under compression—local outliers (20% transfer) and neighborhood-based detection (76% retention)—providing practical guidance for deploying distilled anomaly detectors.
Reliable uncertainty quantification matters for high-stakes tabular classification, yet the comparative influence of architecture and post-hoc calibration on uncertainty quality remains underexplored, particularly outside in-distribution conditions. We present a matched-protocol benchmark on 36 OpenML-CC18 datasets comparing three GBDTs, a single MLP, MC-Dropout, and deep ensembles of three, five, and ten members under five post-hoc calibration methods, with additional evaluation under Gaussian feature shift, symmetric label noise, split-conformal coverage, and a sub-comparison against TabPFN v2. All paired comparisons use Wilcoxon signed-rank tests with Holm corrections within pre-specified research questions at . Using this benchmark, calibrator choice had a larger practical influence on uncertainty metrics than the difference between calibrated GBDTs and five-member neural ensembles. We observe a clear architecture-by-calibrator interaction: Dirichlet calibration was generally strongest on the evaluated GBDTs under low-to-moderate class imbalance, whereas temperature scaling was generally strongest on the evaluated neural models. Under controlled covariate shift, the in-distribution ordering reversed, with multinomial logistic recalibration showing the strongest performance among the tested calibrators on the evaluated neural models under heavy perturbation, and under high class imbalance, the preference for Dirichlet calibration on the evaluated GBDTs weakened. GBDTs were run at recommended defaults rather than tuned per dataset, and the covariate-shift protocol uses synthetic Gaussian noise rather than naturalistic out-of-distribution data; the shift-related findings should be read as directional indicators within this protocol. Calibration and architecture should therefore be selected jointly. The preferred calibrator depends on the model family in-distribution and changes again under perturbation.
Heterogeneous anomaly detection ensembles improve robustness by combining complementary detectors, yet existing approaches often rely on heuristic detector selection, fixed contamination assumptions, and equal weighting. We investigate whether compact ensembles of complementary detectors can outperform substantially larger heterogeneous configurations through diversity-aware weighting and adaptive contamination estimation. Experiments on 22 benchmark datasets show that a compact ensemble of four complementary classical detectors outperforms an eleven-detector ensemble containing deep learning components, while requiring only 13.8% of the computational cost. Across the benchmark, the proposed ensemble variants achieve strong rankings while remaining competitive with the strongest individual detectors (Friedman , ). These findings suggest that detector diversity, rather than ensemble size or architectural complexity, is the primary driver of robust unsupervised anomaly detection performance in resource-constrained environments.