Change detection methods play an indispensable role in remote sensing. Some change detection methods have obtained a fairly good performance by introducing attention mechanism on the basis of the convolutional neural network (CNN), but identifying intricate changes remains difficult. In response to these problems, this article proposes a new model for detecting changes in remote sensing, namely, MTCNet, which combines the advantages of multiscale transformer with the convolutional block attention module (CBAM) to improve the detection quality of different remote sensing images. On the basis of traditional convolutions, the transformer module is introduced to extract bitemporal image features by modeling contextual information. Based on the transformer module, a multiscale module is designed to form a multiscale transformer, which can obtain features at different scales in bitemporal images, thereby identifying the changes we are interested in. Based on the multiscale transformer module, the CBAM is introduced. The CBAM is split into a spatial attention module and a channel attention module, which are applied to the front and back ends of the multiscale transformer, respectively. Spatial information and channel information of feature maps are modeled separately. In this article, the validity and efficiency of the method are verified by a large number of experiments on the LEVIR-CD dataset and the WHU-CD dataset.

The convolutional neural network (CNN) was first proposed in 1960s. Hubel and Wiesel¹ observed for the first time that neurons in the visual cortex were sensitive to moving edge in their experiments on visual cortex cells of cats and proposed the concept of “receptive field.” They further discovered the hierarchical processing mechanism of information in visual cortical pathways, pointing out that simple cells detect location information, and complex cells integrate information stimulated by simple cells. The concept of “receptive field” proposed in Ref. 1 was later introduced into the research work of CNNs. In the 1980s, Fukushima and Miyake² proposed “neocognitron” based on the “receptive field,” which can be regarded as the first implementation of CNNs. Neocognitron decomposes a visual model into several submodels and then processes them on the hierarchical and progressive connected feature planes so that the recognition can be completed even if the object has displacement or slight deformation. Neocognitron is the first artificial neural network based on local connectivity and hierarchical structure among neurons. But at that time, due to the lack of suitable learning algorithm, the network adopted other unsupervised algorithms and was mainly applied to handwritten digit recognition.

As a lightweight deep neural network, MobileNet has fewer parameters and higher classification accuracy. In order to further reduce the number of network parameters and improve the classification accuracy, dense blocks that are proposed in DenseNets are introduced into MobileNet. In Dense-MobileNet models, convolution layers with the same size of input feature maps in MobileNet models are taken as dense blocks, and dense connections are carried out within the dense blocks. The new network structure can make full use of the output feature maps generated by the previous convolution layers in dense blocks, so as to generate a large number of feature maps with fewer convolution cores and repeatedly use the features. By setting a small growth rate, the network further reduces the parameters and the computation cost. Two Dense-MobileNet models, Dense1-MobileNet and Dense2-MobileNet, are designed. Experiments show that Dense2-MobileNet can achieve higher recognition accuracy than MobileNet, while only with fewer parameters and computation cost.

Background: Colonic polyps are more likely to be cancerous, especially those with large diameter, large number and atypical hyperplasia. If colonic polyps cannot be treated in early stage, they are likely to develop into colon cancer. Colonoscopy is easily limited by the operator’s experience, and factors such as inexperience and visual fatigue will directly affect the accuracy of diagnosis. Cooperating with Hunan children’s hospital, we proposed and improved a deep learning approach with global average pooling (GAP) in colonoscopy for assisted diagnosis. Our approach for assisted diagnosis in colonoscopy can prompt endoscopists to pay attention to polyps that may be ignored in real time, improve the detection rate, reduce missed diagnosis, and improve the efficiency of medical diagnosis. Methods: We selected colonoscopy images from the gastrointestinal endoscopy room of Hunan children’s hospital to form the colonic polyp datasets. And we applied the image classification method based on Deep Learning to the classification of Colonic Polyps. The classic networks we used are VGGNets and ResNets. By using global average pooling, we proposed the improved approaches: VGGNets-GAP and ResNets-GAP. Results: The accuracies of all models in datasets exceed 98%. The TPR and TNR are above 96 and 98% respectively. In addition, VGGNets-GAP networks not only have high classification accuracies, but also have much fewer parameters than those of VGGNets. Conclusions: The experimental results show that the proposed approach has good effect on the automatic detection of colonic polyps. The innovations of our method are in two aspects: (1) the detection accuracy of colonic polyps has been improved. (2) our approach reduces the memory consumption and makes the model lightweight. Compared with the original VGG networks, the parameters of our VGG19-GAP networks are greatly reduced.

The remote sensing image change detection methods based on deep learning have made great progress.However, many CNN-based methods persistently face challenges in connecting long-range semantic concepts because of their limited receptive fields. Recently, some methods that combine transformers effectively extract global information by modeling the context in the temporal and spatial domains has been proposed to solve the problem, but they still suffer from both the incorrect identification of "non-semantic changes" and the incomplete and irregular boundary extraction due to the deterioration of local feature details. In response to these inquiries, we propose a novel network, CF-GCN, based on graph convolutional structures for change detection. Specifically, in the encoder and decoder of the network, different projection strategies are employed to construct coordinate space graph convolution and feature interaction graph convolution. The Boundary Perception Module extracts spatial boundary features of shallow layers and enhances boundary perception ability during graph-based information propagation, effectively suppressing the tendency of image boundary information to gradually smooth out. At the same time, the knowledge review module is utilized to form knowledge complementarity between key layers of the network, effectively mitigating the propagation of erroneous knowledge in the deep network. On the LEVIR-CD dataset, the IoU score of CF-GCN is 83.41%, which is 0.35% and 0.39% higher than ChangeStar and DMINet, respectively. On the WHU-CD dataset, the F1 and IoU are as high as 91.83% and 84.90%, which are significantly better than other state-of-the-art networks. The experimental results show that, in addition to CNN and Transformer, the graph-convolution structure approach is expected to be another major research direction for performing fully supervised change detection. Our code and pre-trained models will be available at https://github.com/liucongcharles/CF-GCN.

Because deep neural networks (DNNs) are both memory-intensive and computation-intensive, they are difficult to apply to embedded systems with limited hardware resources. Therefore, DNN models need to be compressed and accelerated. By applying depthwise separable convolutions, MobileNet can decrease the number of parameters and computational complexity with less loss of classification precision. Based on MobileNet, 3 improved MobileNet models with local receptive field expansion in shallow layers, also called Dilated-MobileNet (Dilated Convolution MobileNet) models, are proposed, in which dilated convolutions are introduced into a specific convolutional layer of the MobileNet model. Without increasing the number of parameters, dilated convolutions are used to increase the receptive field of the convolution filters to obtain better classification accuracy. The experiments were performed on the Caltech-101, Caltech-256, and Tubingen animals with attribute datasets, respectively. The results show that Dilated-MobileNets can obtain up to 2% higher classification accuracy than MobileNet.