In the letter, we propose a novel yet effective knowledge distillation scheme which mimics an all-round learning process of an excellent student from the teacher, i.e, knowledge review, knowledge preview, and knowledge correction, to acquire more informative and complementary knowledge. In the newly proposed method, to better leverage comprehensive feature knowledge from the teacher model, we propose Knowledge Review and Knowledge Preview Distillation to amalgamate multi-level features from different intermediate layers in both forward and backward pathways and fully distill them through hierarchical context loss, which greatly improves the student's feature learning efficiency. Moreover, we further present a Response Correction Mechanism to reinforce the prediction of student, which can more fully excavate the student's own knowledge, effectively alleviating the negative influence caused by the knowledge gap between the teacher and the student. We verify the effectiveness of our method with various networks on the CIFAR-100 datasets and the proposed method achieves competitive results compared with other state-of-the-art competitors. The code will be available at https://github.com/kbzhang0505/RPC .
1 January 2025 in IEEE Transactions on Image Processing
Image Super-Resolution (SR) has seen remarkable progress with the emergence of transformer-based architectures. However, due to the high computational cost, many existing transformer-based SR methods limit their attention to local windows, which hinders their ability to model long-range dependencies and global structures. To address these challenges, we propose a novel SR framework named Semantic-Driven Global-Local Fusion Transformer (SGLFT). The proposed model enhances the receptive field by combining a Hybrid Window Transformer (HWT) and a Scalable Transformer Module (STM) to jointly capture local textures and global context. To further strengthen the semantic consistency of reconstruction, we introduce a Semantic Extraction Module (SEM) that distills high-level semantic priors from the input. These semantic cues are adaptively integrated with visual features through an Adaptive Feature Fusion Semantic Integration Module (AFFSIM). Extensive experiments on standard benchmarks demonstrate the effectiveness of SGLFT in producing visually faithful and structurally consistent SR results. The code will be available at https://github.com/kbzhang0505/SGLFT.
Semantic segmentation and boundary detection serve as the foundational stages in the visual language navigation task within embodied intelligence. Nevertheless, existing semantic segmentation and boundary detection methods employed in visual language navigation fail to satisfy the demands for real‐time performance and accuracy under resource‐constrained scenarios. In response to this challenge, this study introduces a dual‐branch network. This architecture is designed to be lightweight while fulfilling the real‐time requirements of the agent and achieving an accuracy that is only marginally lower than that of non‐lightweight state‐of‐the‐art methods. Experimental results demonstrate that this approach achieves a parameter count of 2.7M, FLOPs of 29.93G, and a frame rate of 32.18 FPS.