Knowledge distillation (KD) is a prevalent model compression technique in deep learning, aiming to leverage knowledge from a large teacher model to enhance the training of a smaller student model. It has found success in deploying compact deep models in intelligent applications like intelligent transportation, smart health, and distributed intelligence. Current knowledge distillation methods primarily fall into two categories: offline and online knowledge distillation. Offline methods involve a one-way distillation process, transferring unvaried knowledge from teacher to student, while online methods enable the simultaneous training of multiple peer students. However, existing knowledge distillation methods often face challenges where the student may not fully comprehend the teacher's knowledge due to model capacity gaps, and there might be knowledge incongruence among outputs of multiple students without teacher guidance. To address these issues, we propose a novel reciprocal teacher-student learning inspired by human teaching and examining through forward and feedback knowledge distillation (FFKD). Forward knowledge distillation operates offline, while feedback knowledge distillation follows an online scheme. The rationale is that feedback knowledge distillation enables the pre-trained teacher model to receive feedback from students, allowing the teacher to refine its teaching strategies accordingly. To achieve this, we introduce a new weighting constraint to gauge the extent of students' understanding of the teacher's knowledge, which is then utilized to enhance teaching strategies. Experimental results on five visual recognition datasets demonstrate that the proposed FFKD outperforms current state-of-the-art knowledge distillation methods.

We propose an uplink full-duplex (FD) cooperative non-orthogonal multiple access (NOMA) system, where a dedicated Full-duplex (FD) relay is used to help two uplink users transmitting information to the base station. A novel FD relay transmission mode is proposed, in which the FD relay utilizes successive interference cancellation (SIC) and self-interference cancellation to decode two users symbols at the relay receiving antenna, and the relay transmitting antenna transmits superimposed signal to the base station using superposition coding. In this paper, we also consider a more realistic scenario where selfinterference cancellation is imperfect. Hence, the residual-self interference exists. Moreover, the ergodic sum rate and the outage probability of the proposed system are investigated and the accurate analytical expressions are derived. Simulation results validate that, compared with the uplink half-duplex (HD) cooperative NOMA system and the uplink HD cooperative orthogonal multiple access (OMA) system, our proposed system obtains better performance of the ergodic sum rate and outage probability in the main signal to noise ratio (SNR) regime.