Despite outstanding performance in image recognition, convolutional neural networks (CNNs) do not yet achieve the same impressive results on action recognition in videos. This is partially due to the inability of CNN for modeling long-range temporal structures especially those involving individual action stages that are critical to human action recognition. In this paper, we propose a novel action-stage (ActionS) emphasized spatiotemporal Vector of Locally Aggregated Descriptors (ActionS-STVLAD) method to aggregate informative deep features across the entire video according to adaptive video feature segmentation and adaptive segment feature sampling (AVFS-ASFS). In our ActionSST- VLAD encoding approach, by using AVFS-ASFS, the key frame features are chosen and the corresponding deep features are automatically split into segments with the features in each segment belonging to a temporally coherent ActionS. Then, based on the extracted key frame feature in each segment, a flow-guided warping technique is introduced to detect and discard redundant feature maps, while the informative ones are aggregated by using our exploited similarity weight. Furthermore, we exploit an RGBF modality to capture motion salient regions in the RGB images corresponding to action activity. Extensive experiments are conducted on four public benchmarks - HMDB51, UCF101, Kinetics and ActivityNet for evaluation. Results show that our method is able to effectively pool useful deep features spatiotemporally, leading to state-of-the-art performance for videobased action recognition.

In many microarray studies, classifiers have been constructed based on gene signatures to predict clinical outcomes for various cancer sufferers. However, signatures originating from different studies often suffer from poor robustness when used in the classification of datasets independent from which they were generated from. In this paper, we present an unsupervised feature learning framework by integrating a principal component analysis algorithm and autoencoder neural network to identify different characteristics from gene expression profiles. As the foundation for the obtained features, an ensemble classifier based on the AdaBoost algorithm (PCA-AE-Ada) was constructed to predict clinical outcomes in breast cancer. During the experiments, we established an additional classifier with the same classifier learning strategy (PCA-Ada) in order to perform as a baseline to the proposed method, where the only difference is the training inputs. The AUC (area under the receiver operating characteristic curve) index, MCC (Matthews correlation coefficient) index, ACC (accuracy), and other evaluation parameters of the proposed method were tested on several independent breast cancer datasets and compared with representative gene signature-based algorithms including the baseline method. Experimental results demonstrate that the proposed method using deep learning techniques performs better than others.

Circulating tumor cells (CTCs) detected in peripheral blood (PB) of cancer patients can be identified as isolated CTCs and circulating tumor microemboli (CTM). This study aimed to evaluate the prognostic value of CTM detection and CTC phenotype in advanced colorectal cancer (CRC) patients during chemotherapy. A size-based platform for CTC isolation was applied. PB samples (5 ml) from 98 advanced CRC patients during 2–6 cycles chemotherapy were collected for CTC detection, and CTC count was correlated to patient’s clinicopathological characteristics and clinical outcome. And CTC phenotype was measured by immunofluorescent staining and evaluate the predictive significance on survival in 32 CTCs-positive patients with advanced CRC. Forty-eight of 98 patients were CTCs-positive, including 18 CTM-positive patients, and CTC detection was positively correlated with lymphatic invasion (P = 0.049), TNM stage (P = 0.023), and serum CEA level (P = 0.014). Moreover, Kaplan–Meier survival and Cox regression analyses revealed that the presence of CTCs was an independent factor for poor PFS and OS (P < 0.05) in advanced CRC patients during chemotherapy, and CTM-positive patients had shooter survival than isolated CTCs-positive patients (P < 0.05). Furthermore, patients with vimentin+ isolated CTCs/CTM had shorter PFS and OS compared with CK+ CTCs (P < 0.05). This study provided evidence that the presence of CTCs was positively correlated with poor prognosis, and furthermore, CTM and vimentin+ CTCs predicted poorer survival, which indicated that CTM and vimentin+ CTCs detected by a sensitive platform could be used to improve prognostic value of CTCs in advanced CRC patients under treatment.

Many keywords in a sentence that represent the semantic propensity of the sentence. These words can exist anywhere in the sentence, which poses a great challenge to sentence semantic classification. The current sentence semantic classification methods usually tackle this problem by the use of attention mechanism, and most of them utilize softmax function to calculate each words weight. According to the observation that a word with higher score carries more valuable information in sentence modeling, this paper presents a novel low-complexity model termed as CNN-BiGRU by integrating both Convolution Neural Network (CNN) and Bidirectional Gated Recurrent Unit (BiGRU). Both contextual representations and the semantic distribution are obtained through BiGRU, and the latter is constrained to a Gaussian distribution. In addition, the proposed model utilizes shallow word-level CNN to obtain intermediate representations, and the score of each word is denoted as the Euclidean distance between the intermediate representations and the semantic distribution. Then the final representations are obtained by the combination of the contextual representations and the score of each word, thus the model learns a compact code for sentence sentiment classification and can be trained end-to-end with limited hyper-parameters. In conclusion, the proposed model is able to focus both the keywords and the underlying semantics of the words. Comprehensive experiments are conducted on seven benchmarks. Compared with the state-of-the-art models, our model has excellent performance.

Estimating optical flow from successive video frames is one of the fundamental problems in computer vision and image processing. In the era of deep learning, many methods have been proposed to use convolutional neural networks (CNNs) for optical flow estimation in an unsupervised manner. However, the performance of unsupervised optical flow approaches is still unsatisfactory and often lagging far behind their supervised counterparts, primarily due to over-smoothing across motion boundaries and occlusion. To address these issues, in this paper, we propose a novel method with a new post-processing term and an effective loss function to estimate optical flow in an unsupervised, end-to-end learning manner. Specifically, we first exploit a CNN-based non-local term to refine the estimated optical flow by removing noise and decreasing blur around motion boundaries. This is implemented via automatically learning weights of dependencies over a large spatial neighborhood. Because of its learning ability, the method is effective for various complicated image sequences. Secondly, to reduce the influence of occlusion, a symmetrical energy formulation is introduced to detect the occlusion map from refined bi-directional optical flows. Then the occlusion map is integrated to the loss function. Extensive experiments are conducted on challenging datasets, i.e. FlyingChairs, MPI-Sintel and KITTI to evaluate the performance of the proposed method. The state-of-the-art results demonstrate the effectiveness of our proposed method.

The complexity and diversity of buildings make it challenging to extract low-level and high-level features with strong feature representation by using deep neural networks in building extraction tasks. Meanwhile, deep neural network-based methods have many network parameters, which take up a lot of memory and time in training and testing. We propose a novel fully convolutional neural network called the Context Feature Enhancement Network (CFENet) to address these issues. CFENet comprises three modules: the spatial fusion module, the focus enhancement module, and the feature decoder module. First, the spatial fusion module aggregates the spatial information of low-level features to obtain buildings’ outline and edge information. Secondly, the focus enhancement module fully aggregates the semantic information of high-level features to filter the information of building-related attribute categories. Finally, the feature decoder module decodes the output of the above two modules to segment the buildings more accurately. In a series of experiments on the WHU Building Dataset and the Massachusetts Building Dataset, our CFENet balances efficiency and accuracy compared to the other four methods we compared, and achieves optimality on all five evaluation metrics: PA, PC, F1, IoU, and FWIoU. This indicates that CFENet can effectively enhance and fuse buildings’ low-level and high-level features, improving building extraction accuracy.