CBCT-based synthetic CT generation using generative adversarial networks with disentangled representation Jiwei Liu, Hui Yan, Hanlin Cheng, Jianfei Liu, Pengjian Sun, Boyi Wang, Ronghu Mao, Chi Du, Shengquan Luo

With the development of low-cost RGB-D (Red Green Blue-Depth) sensors, RGB-D object recognition has attracted more and more researchers’ attention in recent years. The deep learning technique has become popular in the field of image analysis and has achieved competitive results. To make full use of the effective identification information in the RGB and depth images, we propose a multi-modal deep neural network and a DS (Dempster Shafer) evidence theory based RGB-D object recognition method. First, the RGB and depth images are preprocessed and two convolutional neural networks are trained, respectively. Next, we perform multi-modal feature learning using the proposed quadruplet samples based objective function to fine-tune the network parameters. Then, two probability classification results are obtained using two sigmoid SVMs (Support Vector Machines) with the learned RGB and depth features. Finally, the DS evidence theory based decision fusion method is used for integrating the two classification results. Compared with other RGB-D object recognition methods, our proposed method adopts two fusion strategies: Multi-modal feature learning and DS decision fusion. Both the discriminative information of each modality and the correlation information between the two modalities are exploited. Extensive experimental results have validated the effectiveness of the proposed method.

For multi-view convolutional neural network based 3D object recognition, how to fuse the information of multiple views is a key factor affecting the recognition performance. Most traditional methods use max-pooling algorithm to obtain the final 3D object feature, which does not take into account the correlative information between different views. To make full use of the effective information of multiple views, this paper introduces the hierarchical graph attention based multi-view convolutional neural network for 3D object recognition. At first, the view selection module is proposed to reduce redundant view information in multiple views, which can select the projective views with more effective information. Then, the correlation weighted feature aggregation module is proposed to better fuse multiple view features. Finally, the hierarchical feature aggregation network structure is designed to further to make full use of the correlation information of multiple views. Extensive experimental results have validated the effectiveness of the proposed method.