Referring video object segmentation (R-VOS) is a fundamental vision-language task which aims to segment the target referred by language expression in all video frames. Existing query-based R-VOS methods have conducted in-depth exploration of the interaction and alignment between visual and linguistic features but fail to transfer the information of the two modalities to the query vector with balanced intensities. Furthermore, most of the traditional approaches suffer from severe information loss in the process of multi-scale feature fusion, resulting in inaccurate segmentation. In this paper, we propose DCT, an end-to-end decoupled cross-modal transformer for referring video object segmentation, to better utilize multi-modal and multi-scale information. Specifically, we first design a Language-Guided Visual Enhancement Module (LGVE) to transmit discriminative linguistic information to visual features of all levels, performing an initial filtering of irrelevant background regions. Then, we propose a decoupled transformer decoder, using a set of object queries to gather entity-related information from both visual and linguistic features independently, mitigating the attention bias caused by feature size differences. Finally, the Cross-layer Feature Pyramid Network (CFPN) is introduced to preserve more visual details by establishing direct cross-layer communication. Extensive experiments have been carried out on A2D-Sentences, JHMDB-Sentences and Ref-Youtube-VOS. The results show that DCT achieves competitive segmentation accuracy compared with the state-of-the-art methods.

This paper proposes a multi-scale referring image segmentation method based on dual attention to solve the problem of insufficient interaction between visual and linguistic modes, as well as different structural and semantic information required by objects of different sizes. Firstly, the dual attention mechanism is used to realize the intermodal and intramodal interaction between vision and text, which enhances the ability to align visual and linguistic features accurately by using different types of information words in the expression. Secondly, using language features as guidance, useful features are selected from other levels for information exchange to further enhance feature representation. Then, dual path ConvLSTM is used to fully integrate low-level visual details and high-level semantics from bottom-up and top-down paths. Finally, multi-scale information is fused by atrous spatial pyramid pooling, increasing the perception ability of the model for different scales. Experiments on the UNC, UNC+, GRef, and ReferIt reference data sets show that the proposed method oIoU improves by 1.81 percentage points on UNC, 1.26 percentage points on UNC+, 0.84 percentage points on GRef, and 0.32 percentage points on ReferIt. Extensive ablation studies have also validated the effectiveness of each component of our approach.