Large-scale deep learning models have achieved impressive results on a variety of tasks; however, their deployment on edge or mobile devices is still a challenge due to the limited available memory and computational capability. Knowledge distillation is an effective model compression technique, which can boost the performance of a lightweight student network by transferring the knowledge from a more complex model or an ensemble of models. Due to its reduced size, this lightweight model is more suitable for deployment on edge devices. In this paper, we introduce an online knowledge distillation framework, which relies on an original attention mechanism to effectively combine the predictions of a cohort of lightweight (student) networks into a powerful ensemble, and use this as a distillation signal. The proposed aggregation strategy uses the predictions of the individual students as well as ground truth data to determine a set of weights needed for ensembling these predictions. This mechanism is solely used during system training. When testing or at inference time, a single, lightweight student is extracted and used. The extensive experiments we performed on several image classification benchmarks, both by training models from scratch (on CIFAR-10, CIFAR-100, and Tiny ImageNet datasets) and using transfer learning (on Oxford Pets and Oxford Flowers datasets), showed that the proposed framework always leads to an improvement in the accuracy of knowledge-distilled students and demonstrates the effectiveness of the proposed solution. Moreover, in the case of ResNet architecture, we observed that the knowledge-distilled model achieves a higher accuracy than a deeper, individually trained ResNet model.

This article describes a reproducible method for automated tooth-level detection and mapping of dental pathologies on panoramic radiographs. The workflow integrates two deep learning stages: (1) segmentation of individual teeth and (2) detection and classification of dental lesions. Outputs are combined to assign pathologies to specific teeth, supporting comprehensive radiological assessment. The method was implemented and tested using two datasets: a collection of 1,628 panoramic radiographs annotated with 12 pathology categories, and the public Tufts Dental Database. Several model architectures were evaluated, including YOLOv11 and RT-DETR configurations. The influence of image formatting and preprocessing strategies on performance was also assessed. Results showed that RT-DETR-x achieved the highest mean average precision (mAP@50) for pathology detection, while YOLOv11x produced the most accurate tooth segmentation. The integrated system demonstrated high precision in linking lesions to the corresponding teeth. This method offers a practical framework for developing AI-assisted diagnostic tools in dentistry and can be adapted to other imaging datasets. • Development of a fully automated dental diagnostic tool that segments individual teeth and maps detected pathologies to their corresponding tooth • YOLOv11 and RT-DETR variants are extensively tested for segmentation and detection tasks • Preprocessing and image formatting strategies are systematically compared.

Accessibility on edge devices and the trade-off between latency and accuracy is an area of interest in deploying deep learning models. This paper explores a Mixture of Experts system, namely, DynK-Hydra, which allows training of an ensemble formed of multiple similar branches on data sets with a high number of classes, but uses, during the inference, only a subset of necessary branches. We achieve this by training a cohort of specialized branches (deep network of reduced size) and a gater/supervisor, that decides dynamically what branch to use for any specific input. An original contribution is that the number of chosen models is dynamically set, based on how confident the gater is (similar works use a static parameter for this). Another contribution is the way we ensure the branches’ specialization. We divide the data set classes into multiple clusters, and we assign a cluster to each branch while enforcing its specialization on this cluster by a separate loss function. We evaluate DynK-Hydra on CIFAR-100, Food-101, CUB-200, and ImageNet32 data sets and we obtain improvements of up to 4.3% accuracy compared with state-of-the-art ResNet. All this while reducing the number of inference flops by a factor of 2–5.5 times. Compared to a similar work (HydraRes), we obtain marginal accuracy improvements of up to 1.2% on the pairwise inference time architectures. However, we improve the inference times by up to 2.8 times compared to HydraRes.