Existing studies on knowledge distillation typically focus on teacher-centered methods, in which the teacher network is trained according to its own standards before transferring the learned knowledge to a student one. However, due to differences in network structure between the teacher and the student, the knowledge learned by the former may not be desired by the latter. Inspired by human educational wisdom, this paper proposes a Student-Centered Distillation (SCD) method that enables the teacher network to adjust its knowledge transfer according to the student network's needs. We implemented SCD based on various human educational wisdom, e.g., the teacher network identified and learned the knowledge desired by the student network on the validation set, and then transferred it to the latter through the training set. To address the problems of current deficiency knowledge, hard sample learning and knowledge forgetting faced by a student network in the learning process, we introduce and improve Proportional-Integral-Derivative (PID) algorithms from automation fields to make them effective in identifying the current knowledge required by the student network. Furthermore, we propose a curriculum learning-based fuzzy strategy and apply it to the proposed PID control algorithm, such that the student network in SCD can actively pay attention to the learning of challenging samples after with certain knowledge. The overall performance of SCD is verified in multiple tasks by comparing it with state-of-the-art ones. Experimental results show that our student-centered distillation method outperforms existing teacher-centered ones.

Knowledge distillation (KD) is a technique that transfers “dark knowledge” from a deep teacher network (teacher) to a shallow student network (student). Despite significant advances in KD, existing work has not adequately mined two crucial types of knowledge: 1) the knowledge of head categories, which represents the relationship between the target category and its similar categories. Our findings reveal that this highly similar (complex) knowledge is essential for improving student’s performance; and 2) the effectively utilized knowledge of tail categories. Existing studies often treat the non-target categories collectively without sufficiently considering the effectiveness of knowledge from tail categories. To tackle these challenges, we reformulate classical KD (ReKD) into two components: Top- K Inter-class Similar Distillation (TISD) and Non-Top- K Inter-class Discriminability (NTID). Firstly, TISD captures and imparts the knowledge of head categories to the student. Our experimental results have verified that TISD is particularly effective in transferring the knowledge of head categories, even in fine-grained dataset classification. Secondly, we theoretically show that the weighting coefficient of NTID increases with the probability of Top- K , leading to stronger suppression of knowledge transfer for tail categories. This observation explains why difficult samples are more informative than simple ones. To better utilize both types of knowledge, we optimize both TISD and NTID using different weighting coefficients, thereby enhancing the student’s ability to learn this valuable knowledge from both head and tail categories. Furthermore, our extensive experimental results demonstrate that ReKD achieves state-of-the-art performance on various image classification datasets, including CIFAR-100, Tiny-ImageNet, and ImageNet-1K, as well as object detection and instance segmentation using the MS-COCO dataset.

Long-landing Events (LLEs) can occur as a result of pilot’s improper operation, resulting in shorter available runways and higher operating costs. The LLE can be effectively pinpointed by analyzing data from the Quick Access Recorder (QAR), which records all of the pilot’s operations during takeoff and landing. Traditionally, domain experts inspect LLEs by manually setting thresholds on uni-dimensional data. However, they cannot detect effectively the defects caused by the pilot’s maneuvering technique because the potential mutual information between different features in the large amount of data is not considered. This paper proposes a data-driven LLE detection and causation analysis workflow, which can automatically mine and analyze the mutual information, to overcome the existing problems. Firstly, a dataset is established based on the extracted QAR data from 2002 flights, considering the landing phase of the aircraft. Subsequently, this paper proposes a Hybrid Feature Selection (HFS) method for selecting features that are highly correlated with LLEs in both supervised and unsupervised ways. A categorical Light Gradient Boosting Machine (LGBM) with a Bayesian optimization (LGBMBO) model is used to determine the performance improvement. Furthermore, the model is visualized to analyze the marginal effect of key parameters for the LLEs by using SHapley Additive exPlanations (SHAP). The experimental results demonstrate that our model reduces computational cost and achieves better performance. Additionally, this paper demonstrates that LLEs can be avoided during the landing phase by maintaining the appropriate descent speed, aircraft altitude, and descent angle.