The reliable operation of photovoltaic (PV) systems is often compromised by surface soiling and structural damage, which reduce energy efficiency and complicate large-scale monitoring. To address this challenge, we propose a two-tiered image-classification framework that combines Vision Transformer (ViT) models, lightweight convolutional neural networks (CNNs), and knowledge distillation (KD). In Tier 1, a DINOv2 ViT-Base model is fine-tuned to provide robust high-level categorization of solar-panel images into three classes: Normal, Soiled, and Damaged. In Tier 2, two enhanced EfficientNetB0 models are introduced: (i) a KD-based student model distilled from a DINOv2 ViT-S/14 teacher, which improves accuracy from 96.7% to 98.67% for damage classification and from 90.7% to 92.38% for soiling classification, and (ii) an EfficientNetB0 augmented with Multi-Head Self-Attention (MHSA), which achieves 98.73% accuracy for damage and 93.33% accuracy for soiling. These results demonstrate that integrating transformer-based representations with compact CNN architectures yields a scalable and efficient solution for automated monitoring of the condition of PV systems, offering high accuracy and real-time applicability in inspections on solar farms.

Knowledge Distillation (KD) is a machine learning technique in which a compact student model learns to replicate the performance of a larger teacher model by mimicking its output predictions. Multi-Teacher Knowledge Distillation extends this paradigm by aggregating knowledge from multiple teacher models to improve generalization and robustness. However, effectively integrating outputs from diverse teachers, especially in the presence of noise or conflicting predictions, remains a key challenge. In this work, we propose a Multi-Round Parallel Multi-Teacher Distillation (MPMTD) that systematically explores and combines multiple aggregation techniques. Specifically, we investigate aggregation at different levels, including loss-based and probability-distribution-based fusion. Our framework applies different strategies across distillation rounds, enabling adaptive and synergistic knowledge transfer. Through extensive experimentation, we analyze the strengths and weaknesses of individual aggregation methods and demonstrate that strategic sequencing across rounds significantly outperforms static approaches. Notably, we introduce the Byzantine-Resilient Probability Distribution aggregation method applied for the first time in a KD context, which achieves state-of-the-art performance, with an accuracy of 99.29% and an F1-score of 99.27%. We further identify optimal configurations in terms of the number of distillation rounds and the ordering of aggregation strategies, balancing accuracy with computational efficiency. Our contributions include (i) the introduction of advanced aggregation strategies into the KD setting, (ii) a systematic evaluation of their performance, and (iii) practical recommendations for real-world deployment. These findings have significant implications for distributed learning, edge computing, and IoT environments, where efficient and resilient model compression is essential.