Finetuning pre-trained deep neural networks (DNN) delicately designed for large-scale natural images may not be suitable for medical images due to the intrinsic difference between the datasets. We propose a strategy to modify DNNs, which improves their performance on retinal optical coherence tomography (OCT) images. Deep features of pre-trained DNN are high-level features of natural images. These features harm the training of transfer learning. Our strategy is to remove some deep convolutional layers of the state-of-the-art pre-trained networks: GoogLeNet, ResNet and DenseNet. We try to find the optimized deep neural networks on small-scale and large-scale OCT datasets, respectively, in our experiments. Results show that optimized deep neural networks not only reduce computational burden, but also improve classification accuracy.
Optical coherence tomography (OCT) is an optical high-resolution imaging technique for ophthalmic diagnosis. In this paper, we take advantages of multi-scale input, multi-scale side output and dual attention mechanism and present an enhanced nested U-Net architecture (MDAN-UNet), a new powerful fully convolutional network for automatic end-to-end segmentation of OCT images. We have evaluated two versions of MDAN-UNet (MDAN-UNet-16 and MDAN-UNet-32) on two publicly available benchmark datasets which are the Duke Diabetic Macular Edema (DME) dataset and the RETOUCH dataset, in comparison with other state-of-the-art segmentation methods. Our experiment demonstrates that MDAN-UNet-32 achieved the best performance, followed by MDAN-UNet-16 with smaller parameter, for multi-layer segmentation and multi-fluid segmentation respectively.
We present an automatic method based on transfer learning for the identification of dry age-related macular degeneration (AMD) and diabetic macular edema (DME) from retinal optical coherence tomography (OCT) images. The algorithm aims to improve the classification performance of retinal OCT images and shorten the training time. Firstly, we remove the last several layers from the pre-trained Inception V3 model and regard the remaining part as a fixed feature extractor. Then, the features are used as input of a convolutional neural network (CNN) designed to learn the feature space shifts. The experimental results on two different retinal OCT images datasets demonstrate the effectiveness of the proposed method.
With a soaring increase in the application of video surveillance in daily life, the estimation of crowd density has already become a hot field. Crowd counting has a very close relationship with traffic planning, pedestrian analysing and emergency warning. Here, a novel crowd counting method based on multi‐scales head detection is proposed. The authors’ approach first uses gradients difference to extract the foreground of the images and apply the overlapped patches in different scales to split the input images. Then, the patches are selected and classified into different groups corresponding to their gradient distributions, and features are extracted for training. Finally, with the predicting result, density maps of different scales are computed and summed with the perspective map. In particular, the authors’ method overcomes the traditional detecting method's deficiencies of low accuracy when facing perspective transformation. Also, experiments demonstrate that this proposed method not only achieved high accuracy in counting but also has outstanding robustness in our data sets.
Modeling spatiotemporal representations is one of the most essential yet challenging issues in video action recognition. Existing methods lack the capacity to accurately model either the correlations between spatial and temporal features or the global temporal dependencies. Inspired by the two-stream network for video action recognition, we propose an encoder–decoder framework named Two-Stream Bidirectional Long Short-Term Memory (LSTM) Residual Network (TBRNet) which takes advantage of the interaction between spatiotemporal representations and global temporal dependencies. In the encoding phase, the two-stream architecture, based on the proposed Residual Convolutional 3D (Res-C3D) network, extracts features with residual connections inserted between the two pathways, and then the features are fused to become the short-term spatiotemporal features of the encoder. In the decoding phase, those short-term spatiotemporal features are first fed into a temporal attention-based bidirectional LSTM (BiLSTM) network to obtain long-term bidirectional attention-pooling dependencies. Subsequently, those temporal dependencies are integrated with short-term spatiotemporal features to obtain global spatiotemporal relationships. On two benchmark datasets, UCF101 and HMDB51, we verified the effectiveness of our proposed TBRNet by a series of experiments, and it achieved competitive or even better results compared with existing state-of-the-art approaches.