Multimodal sentiment analysis (MSA) plays an important role in many applications, such as intelligent question-answering, computer-assisted psychotherapy and video understanding, and has attracted considerable attention in recent years. It leverages multimodal signals including verbal language, facial gestures, and acoustic behaviors to identify sentiments in videos. Language modality typically outperforms nonverbal modalities in MSA. Therefore, strengthening the significance of language in MSA will be a vital way to promote recognition accuracy. Considering that the meaning of a sentence often varies in different nonverbal contexts, combining nonverbal information with text representations is conducive to understanding the exact emotion conveyed by an utterance. In this paper, we propose a Cross-modal Enhancement Network (CENet) model to enhance text representations by integrating visual and acoustic information into a language model. Specifically, it embeds a Cross-modal Enhancement (CE) module, which enhances each word representation according to long-range emotional cues implied in unaligned nonverbal data, into a transformer-based pre-trained language model. Moreover, a feature transformation strategy is introduced for acoustic and visual modalities to reduce the distribution differences between the initial representations of verbal and nonverbal modalities, thereby facilitating the fusion of distinct modalities. Extensive experiments on benchmark datasets demonstrate the significant gains of CENet over state-of-the-art methods. The implementation codes are available on https://github.com/Say2L/CENet.

Cross-modal hashing has become a vital technique in cross-modal retrieval due to its fast query speed and low storage cost in recent years. Generally, most of the priors supervised cross-modal hashing methods are flat methods which are designed for non-hierarchical labeled data. They treat different categories independently and ignore the inter-category correlations. In practical applications, many instances are labeled with hierarchical categories. The hierarchical label structure provides rich information among different categories. To rationally take use of category correlations, hierarchical cross-modal hashing is proposed. However, existing methods intend to preserve instance-pairwise or class-pairwise similarities, which cannot fully explore the semantic correlations among different categories and make the learned hash codes less discriminative. In this paper, we propose a deep cross-modal hashing method named hierarchical semantic structure preserving hashing (HSSPH), which directly exploits the label hierarchy information to learn discriminative hash codes. Specifically, HSSPH learns a set of class-wise hash codes for each layer. By augmenting class-wise codes with labels, it generates layer-wise prototype codes which reflect the semantic structure of each layer. In order to enhance the discriminative ability of hash codes, HSSPH supervises the hash codes learning with both labels and semantic structures to preserve the hierarchical semantics. Besides, efficient optimization algorithms are developed to directly learn the discrete hash codes for each instance and each class. Extensive experiments on two benchmark datasets show the superiority of HSSPH over several state-of-the-art methods.

A multiscale and multidirectional network named the Contourlet convolutional neural network (CCNN) is proposed for synthetic aperture radar (SAR) image despeckling. SAR image resolution is not higher than that of optical images. If the network depth is increased blindly, the SAR image detail information flow will become quite weak, resulting in severe vanishing/exploding gradients. In this paper, a multiscale and multidirectional convolutional neural network is constructed, in which a single-stream structure of convolutional layers is replaced with a multiple-stream structure to extract image features with multidirectional and multiscale properties, thus significantly improving the despeckling performance. With the help of the Contourlet, the CCNN is designed with multiple independent subnetworks to respectively capture abstract features of an image in a certain frequency and direction band. The CCNN can increase the number of convolutional layers by increasing the number of subnetworks, which makes the CCNN not only have enough convolutional layers to capture the SAR image features, but also overcome the problem of vanishing/exploding gradients caused by deepening the networks. Extensive quantitative and qualitative evaluations of synthetic and real SAR images show the superiority of our proposed method over the state-of-the-art speckle reduction method.

Visual grounding of remote sensing (RSVG) is a task to locate targets indicated by referring expressions in remote sensing (RS) images. Previous approaches directly concatenate visual and language features and stack a series of transformer encoders for cross-modal fusion. However, this fusion strategy fails to fully leverage attributes and contextual information of the targets in referring expressions, limiting the performance of the existing methods. To address this issue, we propose a novel visual grounding framework for RSVG, named VSMR, which achieves accurate localization by adaptively selecting target-relevant features and performing multistage cross-modal reasoning. Specifically, we propose an adaptive feature selection (AFS) module, which automatically selects visual features relevant to queries while suppressing background noises. A multistage decoder (MSD) is designed to iteratively infer correlations between images and queries by leveraging abundant object attributes and contextual information in the referring expressions, thereby achieving accurate target localization. Experiments demonstrate that our method is superior to other state-of-the-art (SoTA) methods, achieving an accuracy of 78.24%.

Edge computing provides capabilities similar to traditional cloud computing at network edge closer to users. Offloading applications from the cloud to the edge of the network can effectively diminish the latency of users waiting for applications to improve their quality of service(QoS). Due to the user mobility, however, the distance between the user and the edge device is constantly increasing, resulting in a degradation of the user’s QoS. To satisfy the QoS of mobile users, applications should be dynamically migrated to follow the user mobility. In this paper, we propose a mobility-aware dynamic service placement policy that reduces the number of virtual machine migrations and diminishes user-perceived latency simultaneously. The proposed scheme filters out the invalid migration that the user handoffs to another access point before the migration is completed. The simulation trials demonstrate that our solution improves the QoS of users and reduces the number of application migrations.