Wei Xu

Wei Xu

Auto-generated profile, based on publicly available information

Wei Xu

Nanjing University of Posts and Telecommunications, Nanjing, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 June 2024 in IEEE MultiMedia
Verified
Journal Article
Closed access
NSNing Sun
Ning Sun
WXWei Xu
Wei Xu
JLJixin Liu
Jixin Liu
LCLei Chai
Lei Chai
HSHaian Sun
Haian Sun
Scene recognition is a challenging task in computer vision because of the diversity of objects in scene images and the ambiguity of object layouts. In recent years, the emergence of multimodal scene data has provided new solutions for scene recognition, but it has also brought new problems. To address these challenges, the Self-Attention and Distillation based Multimodal Scene Recognition Network (SAD-MSR) is proposed in this paper. The backbone of the model adopts the pure transformer structure of self-attention, which can extract local and global spatial features of multimodal scene images. A multi-stage fusion mechanism was developed for this model, in which the concatenated tokens of two modalities are fused based on self-attention in the early stage, while the high-level features extracted from the two modalities are fused based on cross-attention in the late stage. Furthermore, a distillation mechanism is introduced to alleviate the problem of a limited number of training samples. Finally, we conducted extensive experiments on two multimodal scene recognition databases, SUN RGB-D and NYUD2, to show the effectiveness of SAD-MSR. Compared with other state-of-the-art multimodal scene recognition methods, our method can achieve better experimental results.