Capsule Network is a novel and promising neural network in the field of deep learning, which has shown good performance in image classification by encoding features into capsules and constructing the part-whole relationships. However, the original Capsule Network is not suitable for the images with complex background due to its weak ability of feature extraction, a large number of training parameters, and the characteristic of explaining everything in the image. To address the above issues, we propose an advanced Capsule Network named RS-CapsNet, which uses Res2Net block to extract multi-scale features, and Squeeze-and-Excitation (SE) block to highlight useful features and suppress useless features. Meanwhile, we adopt the method of linear combination between capsules, which enhances the representation ability of capsules for detected object and reduces the number of capsules. Moreover, we propose the method of firstly constructing intermediate capsules that can represent most of the detected object, and then using the intermediate capsules and primary capsules to construct the classification capsules together. The experimental results show that proposed RS-CapsNet has better performance on CIFAR10, CIFAR100, SVHN, FashionMNIST, and AffNIST datasets, it can also provide better translation equivariance, and the number of training parameters is reduced by 65.11%.

Convolutional neural networks (CNNs) have achieved great success in visual recognition because of the availability of large-scale image datasets, such as the ImageNet. The transfer of convolutional features to challenging scene recognition remains an open problem. Multiple non-linear transforms endow the convolutional features with abundant information. On the other side, CNNs are adept at capturing the holistic appearances of scenes, whereas the lack of some critical local details may reduce the recognition accuracy. To address these problems, we propose a novel hierarchical coding algorithm to learn effective representations. To adapt the scale variations, many useful patches with various scales sampled from the whole image are considered to provide the sufficient details. Non-negative sparse decomposition model (NNSD) based on convolutional features is proposed to learn the sharable components for each scale and further produce global signatures. Based on the global signatures, inter-class linear coding (ICLC) is proposed to learn the discriminative components and ultimate image representations. Experimental results indicate that our approach significantly improves the recognition accuracy compared with general CNN models and achieves excellent performance on five standard benchmarks.