Multi-modal medical images provide complementary soft-tissue characteristics that aid in the screening and diagnosis of diseases. However, limited scanning time, image corruption and various imaging protocols often result in incomplete multi-modal images, thus limiting the usage of multi-modal data for clinical purposes. To address this issue, in this paper, we propose a novel unified multi-modal image synthesis method for missing modality imputation. Our method overall takes a generative adversarial architecture, which aims to synthesize missing modalities from any combination of available ones with a single model. To this end, we specifically design a Commonality- and Discrepancy-Sensitive Encoder for the generator to exploit both modality-invariant and specific information contained in input modalities. The incorporation of both types of information facilitates the generation of images with consistent anatomy and realistic details of the desired distribution. Besides, we propose a Dynamic Feature Unification Module to integrate information from a varying number of available modalities, which enables the network to be robust to random missing modalities. The module performs both hard integration and soft integration, ensuring the effectiveness of feature combination while avoiding information loss. Verified on two public multi-modal magnetic resonance datasets, the proposed method is effective in handling various synthesis tasks and shows superior performance compared to previous methods.

As an intersection of artificial intelligence and human communication analysis, Emotion Recognition in Conversation (ERC) has attracted much research attention in recent years. Existing studies, however, are limited in adequately exploiting latent relations among the constituent utterances. In this paper, we address this issue by proposing a novel approach named Latent Relation-Aware Graph Convolutional Network (LR-GCN), where both speaker dependency of the interlocutors is leveraged and latent correlations among the utterances are captured for ERC. Specifically, we first establish a graph model to incorporate the context information and speaker dependency of the conversation. Afterward, the multi-head attention mechanism is introduced to explore the latent correlations among the utterances and generate a set of all-linked graphs. Here, aiming to simultaneously exploit the original modeled speaker dependency and the explored correlation information, we introduce a dense connection layer to capture more structural information of the generated graphs. Through a multi-branch graph network, we achieve a unified representation of each utterance for final prediction. Detailed evaluations on two benchmark datasets demonstrate LR-GCN outperforms the state-of-the-art approaches.

Due to the wide applications in a rapidly increasing number of different fields, 3D shape recognition has become a hot topic in the computer vision field. Many approaches have been proposed in recent years. However, there remain huge challenges in two aspects: exploring the effective representation of 3D shapes and reducing the redundant complexity of 3D shapes. In this paper, we propose a novel deep-attention network (DAN) for 3D shape representation based on multiview information. More specifically, we introduce the attention mechanism to construct a deep multiattention network that has advantages in two aspects: 1) information selection, in which DAN utilizes the self-attention mechanism to update the feature vector of each view, effectively reducing the redundant information, and 2) information fusion, in which DAN applies attention mechanism that can save more effective information by considering the correlations among views. Meanwhile, deep network structure can fully consider the correlations to continuously fuse effective information. To validate the effectiveness of our proposed method, we conduct experiments on the public 3D shape datasets: ModelNet40, ModelNet10, and ShapeNetCore55. Experimental results and comparison with state-of-the-art methods demonstrate the superiority of our proposed method. Code is released on https://github.com/RiDang/DANN.

2D image-based 3D object retrieval is a novel but challenging task for 3D object retrieval. In this paper, we propose a 2D image-based 3D object retrieval method via joint heterogeneous feature learning and distribution alignment. Specifically, we propose to learn a mapping function in the Grassmann manifold to reduce the divergence of heterogeneous features of 2D images and 3D objects. We further employ the data distribution alignment method to adaptively integrate both marginal and conditional distributions. We embed both terms into the objective function to learn a domain-invariant classifier based on structural risk minimization. The output domain-invariant features of 2D images and 3D objects can be utilized for 2D image-based 3D object retrieval. Since there is lack of large-scale dataset for the evaluation of this task, we build two new datasets, MI3DOR and MI3DOR-2. We compare the proposed method against the representative methods for domain adaption and explore the influence of different components of the objective functions and key parameter. Comparison experiments show the superiority of this method.