With the rapid development of social media, visual sentiment analysis from image or video has become a hot spot in visual understanding researches. In this work, we propose an effective approach using visual and textual fusion for sentiment analysis of short GIF videos with textual descriptions. We extract both sequence-level and frame-level visual features for each given GIF video. Next, we build a visual sentiment classifier by using the extracted features. We also define a mapping function, which converts the sentiment probability from the classifier to a sentiment score used in our fusion function. At the same time, for the accompanied textual annotations, we employ the Synset forest to extract the sets of the meaningful sentiment words and utilize the SentiWordNet3.0 model to obtain the textual sentiment score. Then, we design a joint visual-textual sentiment score function weighted with visual sentiment component and textual sentiment one. To make the function more robust, we introduce a noticeable difference threshold to further process the fused sentiment score.Finally, we adopt grid search technique to obtain the relevant model hyper-parameters by optimizing a sentiment aware score function.Experimental results and analysis extensively demonstrate the effectiveness of the proposed sentiment recognition scheme on three benchmark datasets including T-GIF dataset, GSO-2016 dataset and Adjusted-GIFGIF dataset.

Locating anatomical landmarks in a cephalometric X-ray image is a crucial step in cephalometric analysis. Manual landmark localization suffers from inter-and intra-observer variability, which makes developing automated localization methods urgent in clinics. Most of the existing techniques follow the routine thoughts which estimate numerical values of displacements or coordinates for the target landmarks. Additionally, there are no reported applications of generative adversarial networks (GAN) in cephalometric landmark localization. Motivated by these facts, we propose a new automated cephalometric landmark localization method under the framework of GAN. The principle behind our approach is fundamentally different from the conventional ones. It trained an adversarial network under the framework of GAN to learn the mapping from features to the distance map of a specific target landmark. Namely, the output of the adversarial network in this paper is image data, instead of displacements or coordinates as the conventional approaches. Based on the trained networks, we can predict the distance maps of all target landmarks in a new cephalometric image. Subsequently, the target landmarks are detected from the predicted distance maps by an approach similar to regression voting. Experimental results validate the good performance of our method in localization of cephalometric landmarks in dental X-ray images.

Medical image segmentation remains a challenging task due to the intricate nature of anatomical structures and the wide range of target sizes. In this paper, we propose a novel U -shaped segmentation network that integrates CNN and Transformer architectures to address these challenges. Specifically, our network architecture consists of three main components. In the encoder, we integrate an attention-guided multi-scale feature extraction module with a dual-path downsampling block to learn hierarchical features. The decoder employs an advanced feature aggregation and fusion module that effectively models inter-dependencies across different hierarchical levels. For the bottleneck, we explore multi-scale feature activation and multi-layer context Transformer modules to facilitate high-level semantic feature learning and global context modeling. Additionally, we implement a multi-resolution input-output strategy throughout the network to enrich feature representations and ensure fine-grained segmentation outputs across different scales. The experimental results on diverse multi-modal medical image datasets (ultrasound, gastrointestinal polyp, MR, and CT images) demonstrate that our approach can achieve superior performance over state-of-the-art methods in both quantitative measurements and qualitative assessments. The code is available at https://github.com/zsj0577/MSAGHNet.

Limiting scan views is an efficient way to reduce radiation doses in the cone-beam computed tomography (CBCT) examinations, which unfortunately degrades the reconstructed images. Some methods on the framework of the generative adversarial network (GAN) were developed to improve low-dose CT images after CT reconstruction from the limited-view projections. However, no GAN-based methods were devoted to restoring missing CBCT projections in the sinogram domain before CT reconstruction. To avoid the trade-off between radiation dose and image quality, we propose a limited-view CBCT reconstruction method in the sinogram domain, instead of the image domain. First, this method slices the 3D CBCT projections into multiple 2D pieces. Then, an adversarial autoencoder network is trained to estimate the missing parts of these 2D pieces. To improve the prediction, we apply a joint loss function, including reconstruction loss and adversarial loss to the network. When the new limited-view 3D CBCT projections are acquired, the proposed method uses the trained adversarial autoencoder network to generate the missing parts of the 2D pieces sliced from the current 3D CBCT projections. Then, stacking the completed 2D pieces in order yields full-view 3D CBCT projections. Finally, we reconstruct the CT images from the full-view 3D CBCT projections by using the Feldkamp, Davis, and Kress algorithm. The experiments validate that our method performs well in the prediction of unknown projections and CT reconstruction and are less vulnerable to the number of unknown projections than other methods.