Bi-directional image-text retrieval and matching attract much attention recently. This cross-domain task demands a fine understanding of both modalities for learning a measure of different modality data. In this paper, we propose a novel position focused attention network to investigate the relation between the visual and the textual views. This work integrates the prior object position to enhance the visual-text joint-embedding learning. The image is first split into blocks, which are treated as the basic position cells, and the position of an image region is inferred. Then, we propose a position attention to model the relations between the image region and position cells. Finally, we generate a valuable position feature to further enhance the region expression and model a more reliable relationship between the visual image and the textual sentence. Experiments on the popular datasets Flickr30K and MS-COCO show the effectiveness of the proposed method. Besides the public datasets, we also conduct experiments on our collected practical large-scale news dataset (Tencent-News) to validate the practical application value of the proposed method. As far as we know, this is the first attempt to test the performance on the practical application. Our method achieves the competitive performance on all of these three datasets.
15 May 2023 in IEEE Transactions on Circuits and Systems for Video Technology
We propose a new approach, called Tensor-based Incomplete Multi-view Clustering with Low-rank data Reconstruction and Consistency guidance (TIMC-RC), to perform clustering on multi-view data with missing views. Existing methods usually leverage original incomplete data to explore the partial correlations among multiple views, and do not make sufficient use of both consistent and complementary information across views. To explore the full information of missing and available views, TIMC-RC introduces low-rank data reconstruction and consistency view establishment. Specifically, 1) it adopts a low-rank constraint to reconstruct data representations so as to reduce the negative effect of missing data and obtain more reasonable data representations. 2) It builds a new consistency view by self-representation matrices and therefore explores the consistent correlation of different views. 3) It formalizes view-specific self-representation matrices and the consistent matrix as a tensor and utilizes the tensor singular value decomposition-based nuclear norm to enhance the consistency and complementarity of multi-view representations. Experiments conducted on eight benchmarks verify the effectiveness and advancement of the proposed TIMC-RC.
Aspect-based sentiment analysis aims at identifying the sentiment polarity of specific aspect in the sentence. Previous work has realized the importance of the information interaction between aspect term and context. However, most existing information interaction methods are coarse-grained, which results in a certain loss of information. In addition, most methods ignore the role of position information in identifying the sentiment polarity of the aspect. To better address the two problems, we propose a novel approach, called hierarchical gated deep memory network with position-aware. Our approach has two characteristics: 1) it has fine-grained information interaction attention mechanism which models the wordlevel interaction between aspect and context. The sentence-to-aspect attention is used to capture the most indicative sentiment words in context. And the aspect-to-sentence attention is used to capture the most important word in the aspect term. 2) The position information is embedded as a feature in the sentence representation. Finally, we conduct sentiment classification comparative experiment on laptop and restaurant datasets. The experimental results show that our model achieves state-of-the-art performance on aspect-based sentiment analysis.