Channel pruning enables model acceleration by removing channels from convolutional neural networks (CNNs). However, many existing methods adopt a “hard removal” strategy that directly removing low-importance channels, leading to severe feature loss and accuracy degradation. To address this issue, we propose Reconstruction and Consolidation Pruning (RCP), a pruning framework that decouples the pruning process into a pruning-training phase and an inference phase. During pruning training, RCP generates a pruning strategy based on channel importance under a global pruning rate constraint, and constructs a feature reloading mechanism. This mechanism utilizes a learnable $1\times 1$ compensation convolution to adaptively transfer and fuse discriminative features hidden in the pruned channels into the retained channels. In the inference phase, RCP adopts a linear reparameterization strategy to seamlessly consolidate the compensation branches into the main network branch without loss of performance, ensuring zero additional operator overhead during inference. This reversible structural transformation ensures that the training-time augmented architecture and the inference-time compact architecture are functionally identical under linear consolidation. Experimental results show that at 50% FLOPs reduction, RCP incurs only a 0.84% accuracy drop on ResNet-50 (ImageNet-1K), while at 53% FLOPs reduction it achieves a 0.07% accuracy improvement for ResNet-56 (CIFAR-10), validating the proposed method’s effectiveness and superiority under high compression rates.

Knowledge distillation achieves model compression by training a lightweight student network to mimic the output distribution of a larger teacher network. However, when the teacher becomes overconfident, its sharply peaked logits break the scale symmetry of supervision and induce high-variance gradients, leading to unstable optimization. Meanwhile, research that focuses only on final-logit alignment often fails to utilize intermediate semantic structure effectively. This causes weak discrimination of student representations, especially under class imbalance. To address these issues, we propose Scale Calibration and Pressure-Driven Knowledge Distillation (SPKD): a one-stage framework comprising two lightweight, complementary mechanisms. First, a dynamic scale calibration module normalizes the teacher’s logits to a consistent magnitude, reducing gradient variance. Secondly, an adaptive pressure-driven mechanism refines student learning by preventing feature collapse and promoting intra-class compactness and inter-class separability. Extensive experiments on CIFAR-100 and ImageNet demonstrate that SPKD achieves superior performance to distillation baselines across various teacher–student combinations. For example, SPKD achieves a score of 74.84% on CIFAR-100 for the homogeneous architecture VGG13-VGG8. Additional evidence from logit norm and gradient variance statistics, as well as representation analyses, proves the fact that SPKD stabilizes optimization while learning more discriminative and well-structured features.