Image classification is a fundamental task in computer vision, with convolutional neural networks (CNNs) being the state-of-the-art approach for this task. However, CNNs can be large and computationally expensive, which limits their deployment on resource-constrained devices. To address this issue, model optimization and compression techniques such as knowledge distillation and channel attention have been proposed. In this paper, we propose a modified version of the skip-connection knowledge distillation method, which integrates the efficient channel attention module. Our experiments on the CIFAR-10 dataset show that this approach improves the performance of both large teacher-small student and self-distillation scenarios, and also leads to smaller model sizes. We also visualize the intermediate feature maps to understand the learning process of the network. Our results demonstrate the effectiveness of combining knowledge distillation and channel attention for improving the performance and efficiency of CNNs.

Reflection removal from a single image is an ill-posed problem due to the inherent ambiguity in separating transmission and reflection components from a single composite observation. In this paper, we address this challenge by introducing a reversible feature encoding strategy combined with a simplified dual-stream decoding structure. In particular, the reversible NAFNet encoder enables us to retain all feature information throughout the encoding process while avoiding memory overhead, an aspect that is crucial for separating overlapping structures. In place of complex gated mechanisms, the proposed dual-stream decoder leverages shared encoder features and skip connections, thus enabling implicit bidirectional information flow between transmission and reflection streams. Although our model adopts a lightweight structure and omits attention modules, it achieves competitive results on standard reflection removal benchmarks, indicating that efficient and interpretable designs can match or surpass more complex counterparts.