Medical image registration is crucial in medical image analysis applications. Recently, U-Net-style networks have been commonly used for unsupervised image registration, predicting dense displacement fields in full-resolution space. However, this process is resourceintensive and time-consuming for high-resolution volumetric image data. To address this challenge, this paper proposes a novel model named RegFSC-Net, which utilizes Fourier transform with spatial reorganization (SR) and channel refinement (CR) network for registration. We embed efficient feature extraction modules SR and CR modules into the encoder, and adopt a parameter-free model to drive the decoder to improve the U-shaped network. Precisely, RegFSC-Net does not directly predict the fullresolution displacement field in space but learns the lowdimensional representation of the displacement field in the bandlimited Fourier domain, which is beneficial in reducing network parameters, memory usage, and computational costs. Experimental results show that RegFSC-Net outperforms various state-of-the-art methods. Specifically, in comparison to the widely recognized Transformer-based method TransMorph, RegFSC-Net utilizes only around 8.2% of its parameters, resulting in a 1.95% higher Dice score and significantly faster inference speeds of 126.67% and 419.99% on GPU and CPU, respectively. Furthermore, we also designed three variants of RegFSC-Net and demonstrated their potential applications in computer-aided diagnosis.
Visual sentiment analysis is in great demand as it provides a computational method to recognize sentiment information in abundant visual contents from social media sites. Most of existing methods use CNNs to extract varying visual attributes for image sentiment prediction, but they failed to comprehensively consider the correlation among visual components, and are limited by the receptive field of convolutional layers as a result. In this work, we propose a visual semantic correlation network VSCNet, a Transformer-based visual sentiment prediction model. Precisely, global visual features are captured through an extended attention network stacked by a well-designed extended attention mechanism like Transformer. An off-the-shelf object query tool is used to determine the local candidates of potential affective regions, by which redundant and noisy visual proposals are filtered out. All candidates considered affective are embedded into a computable semantic space. Finally, a fusion strategy integrates semantic representations and visual features for sentiment analysis. Extensive experiments reveal that our method outperforms previous studies on 5 annotated public image sentiment datasets without any training tricks. More specifically, it achieves 1.8% higher accuracy on FI benchmark compared with other state-of-the-art methods.
Attention mechanism has shown excellent performance in many computer vision tasks, while the previous literature may not adequately consider different types of attention mechanisms or is individual elaborate designed for a certain network. In this paper, a general yet effective multipath attention mechanism (MAM) to explore the effect of visual attention for image recognition is proposed. In contrast with other attentions that leverage global pooling, the main advantage is that the MAM considers both the correlation of featuremaps and different scale structural information into account. The backbone representations are enhanced by adding MAM laterally along independent and separate dimensions, channel and spatial. Due to only a simple and unified calculation block is generated, MAM can be flexibly integrated into various CNNs within few parameters and trained together end‐to‐end. Furthermore, the topology structures of attention path arrangement are investigated using different connection schemes. Experimental results on several image recognition datasets show that the model outperforms various existing models. Finally, performance improvement through visualisation is intuitively discussed. The source code for the proposed attention module is publicly available.
Multi‐object tracking (MOT), which uses the context information of image sequences to locate, maintain identities and generate trajectories of multiple targets in each frame, is key technology in the field of computer vision. To address the problems of occlusion and scale variation in low‐viewpoint MOT, OsaMOT is proposed here. First, according to the global occlusion state of each frame, OsaMOT proposes the adaptive anti‐occlusion feature to enhance the awareness and adaptability for occlusion. At the same time, OsaMOT uses the cascade screening mechanism to reduce the “virtual new target” phenomenon due to the dramatic change in target features caused by scale variation and occlusion. Finally, considering that the occluded templates will affect the tracking performance, OsaMOT proposes an adaptive anti‐noise template update mechanism according to the partial occlusion state of the target, which improves the purity of the template library and further enhances the applicability to occlusion. The experimental results show that OsaMOT can weaken the influence of scale variation, partial occlusion, short‐term full occlusion and long‐term full occlusion in the low‐viewpoint tracking scenes. Most evaluation indexes of OsaMOT under low‐viewpoint tracking scenario are superior to those of some typical algorithms proposed in recent years, and the tracking robustness is improved.