As an information carrier with rich semantics, images contain more sentiment than texts and audios. So, images are increasingly used by people to express their opinions and sentiments in social network. The sentiments of the images are overall and should come from different regions. So, the recognition of the sentiment regions will help to concentrate on important factors the affect the sentiments. Meanwhile, deep learning method for image sentiment classification needs simple and efficient approach for simultaneously carrying out pruning and feature selection whilst optimizing the weights. Motivated by these observations, we design a region-based convolutional neural network using group sparse regularization for image sentiment classification: R-CNNGSR. The method obtains the initial sentiment prediction model through CNN using group sparse regularization to get compact neural network, and then automatically detect the sentiment regions by combining the underlying features and sentimental features. Finally, the whole image and the sentiment region are fused to predict the overall sentiment of the images. Experiment results demonstrate that our proposed R-CNNGSR significantly outperforms the state-of-the-art methods in image sentiment classification.

In recent years, image style transfer has been greatly improved by using deep learning technology. However, when directly applied to clothing style transfer, the current methods cannot allow the users to self-control the local transfer position of an image, such as separating specific T-shirt or trousers from a figure, and cannot achieve the perfect preservation of clothing shape. Therefore, this paper proposes an interactive image localized style transfer method especially for clothes. We introduce additional image called outline image, which is extracted from content image by interactive algorithm. The interaction consists simply of dragging a rectangle around the desired clothing. Then, we introduce an outline loss function based on distance transform of the outline image, which can achieve the perfect preservation of clothing shape. In order to smooth and denoise the boundary region, total variation regularization is employed. The proposed method constrains that the new style is generated only in the desired clothing part rather than the whole image including background. Therefore, in our new generated images, the original clothing shape can be reserved perfectly. Experiment results show impressive generated clothing images and demonstrate that this is a good approach to design clothes.

Natural language generation (NLG) is a core component of machine translation, dialogue systems, speech recognition, summarization, and so forth. The existing text generation methods tend to be based on recurrent neural language models (NLMs), which generate sentences from encoding vector. However, most of these models lack explicit structured representation for text generation. In this work, we introduce a new generative model for NLG, called Tree-VAE. First it samples a sentence from the training corpus and then generates a new sentence based on the corresponding parse tree embedding vector. Tree-LSTM is used in collaboration with the Stanford Parser to retrieve sentence construction data, which is then used to train a conditional discretization autoencoder generator based on the embeddings of sentence patterns. The proposed model is extensively evaluated on three different datasets. The experimental results proved that the proposed model can generate substantially more diverse and coherent text than existing baseline methods.

Purpose: The occurrence and dissemination of hate videos in social media platform could pose serious harm to both society and individuals. However, the characteristics of the hate videos increase the difficulty of detection task. Hate content is usually presented in a relatively covert manner in videos, and textual content in videos plays an important role in hate video detection. In this work, we propose a textual context enhanced dynamic bimodal fusion (TCE-DBF) method for hate video detection.Design/methodology/approach: The proposed method TCE-DBF introduces dynamic modality gate (DMG) and bimodal fusion transformer network to dynamically integrate multimodalities. Moreover, in order to enhance textual modality in videos, two types of textual context from the video are taken as the input of TCE-DBF. One is extracted from video frames in visual modality. The other is extracted from audio in acoustic modality. Specially, TCE-DBF splits the original audio and learns the sequence representation to capture acoustic temporal information.Findings: Hate video detection has been one of the hotspots in recent works. However, it still faces two serious challenges. The first challenge is the hate content in videos presented in multimodalities. The second challenge is how to evaluate the importance of different modalities for multimodal fusion modeling. TCE-DBF aims to tackle these challenges. Experimental results on hate video dataset HateMM demonstrate that TCE-DBF outperforms the state-of-the-art methods, and the visualization results show that textual modality plays a more important role in hate video detection. Therefore, it is vital to consider the text in videos.Originality/value: TCE-DBF can be utilized to effectively detect hate videos on social media. Besides transcript, TCE-DBF considers text in video frames, which makes detection more accurate. Meanwhile, to better achieve multimodal fusion, TCE-DBF uses DMG and bimodal fusion transformer network to dynamically assign different weights to three modalities and integrate them. The proposed TCE-DBF is novel in terms of capturing multimodal features, enhancing the textual modality and achieving high detection performance for hate video detection.