Sumesh Prasad

Sumesh Prasad

Auto-generated profile, based on publicly available information

Sumesh Prasad

Infosys Centre for Artificial Intelligence, IIIT Delhi, New Delhi, Delhi, India

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 December 2025
Verified
Conference Paper
Open access
SPSumesh Prasad
Sumesh Prasad
OSOjaswini Sharma
Ojaswini Sharma
SASaket Anand
Saket Anand
Knowledge distillation (KD) transfers what a large, accurate teacher model has learned to a smaller, faster student model. Standard losses such as Kullback–Leibler (KL) divergence, mean squared error (MSE), and dot product have been used successfully. Still, they often come with limitations, for example, assuming specific data distributions or requiring that teacher and student features have the same dimensionality. We introduce Distance Correlation (DC) as an additional distillation loss to get around these issues. DC measures non-linear dependencies between teacher and student representations and works without the need for matching dimensions, making it more flexible. We tested this idea on CIFAR-100 and Tiny ImageNet classification tasks, with Vision Transformers and ResNets as teachers and MobileNetV2 as the students. For object detection, we use a subset of MS COCO with YOLOv8-medium as the teacher and YOLOv8n as the student. Across these settings, combining KL divergence with DC consistently improves results up to a +5.6% boost in classification accuracy and +1.4% mAP in detection over the baseline. These gains suggest that DC is a practical, general purpose addition to existing KD objectives.