Remote sensing image scene classification is one of the most challenging problems in understanding high-resolution remote sensing images. Deep learning techniques, especially the convolutional neural network (CNN), have improved the performance of remote sensing image scene classification due to the powerful perspective of feature learning and reasoning. However, several fully connected layers are always added to the end of CNN models, which is not efficient in capturing the hierarchical structure of the entities in the images and does not fully consider the spatial information that is important to classification. Fortunately, capsule network (CapsNet), which is a novel network architecture that uses a group of neurons as a capsule or vector to replace the neuron in the traditional neural network and can encode the properties and spatial information of features in an image to achieve equivariance, has become an active area in the classification field in the past two years. Motivated by this idea, this paper proposes an effective remote sensing image scene classification architecture named CNN-CapsNet to make full use of the merits of these two models: CNN and CapsNet. First, a CNN without fully connected layers is used as an initial feature maps extractor. In detail, a pretrained deep CNN model that was fully trained on the ImageNet dataset is selected as a feature extractor in this paper. Then, the initial feature maps are fed into a newly designed CapsNet to obtain the final classification result. The proposed architecture is extensively evaluated on three public challenging benchmark remote sensing image datasets: the UC Merced Land-Use dataset with 21 scene categories, AID dataset with 30 scene categories, and the NWPU-RESISC45 dataset with 45 challenging scene categories. The experimental results demonstrate that the proposed method can lead to a competitive classification performance compared with the state-of-the-art methods.

High-resolution remote sensing image-based land-use scene classification is a difficult task, which is to recognize the semantic category of a given land-use scene image based on priori knowledge. Land-use scenes often cover multiple land-cover classes or ground objects, which makes a scene very complex and difficult to represent and recognize. To deal with this problem, this paper applies the well-known bag-of-visual-words (BOVWs) model which has been very successful in natural image scene classification. Moreover, many existing BOVW methods only use scale-invariant feature transform (SIFT) features to construct visual vocabularies, lacking in investigation of other features or feature combinations, and they are also sensitive to the rotation of image scenes. Therefore, this paper presents a concentric circle-based spatial-rotation-invariant representation strategy for describing spatial information of visual words and proposes a concentric circle-structured multiscale BOVW method using multiple features for land-use scene classification. Experiments on public land-use scene classification datasets demonstrate that the proposed method is superior to many existing BOVW methods and is very suitable to solve the land-use scene classification problem.

To obtain a complete representation of scene information in high spatial resolution remote sensing scene images, an increasing number of studies have begun to pay attention to the multiple low-level feature types-based bag-of-visual-words (multi-BOVW) model, for which the two-phase classification-based multi-BOVW method is one of the most popular approaches. However, this method ignores the information of feature significance among different feature types in the score-level fusion stage, thus affecting the classification performance of the multi-BOVW methods. To address this limitation, a feature significance-based multi-BOVW scene classification method was proposed, which integrates the information of feature separating capabilities among different scene categories into the traditional two-phase classification-based score-level fusion framework, realizing different treatments for different feature channels in classifying different scene categories. Experimental results show that the proposed method outperforms the traditional score-level fusion-based multi-BOVW methods and effectively explores the feature significance information in multiclass remote sensing image scene classification tasks.

Previous works about spatial information incorporation into a traditional bag-of-visual-words (BOVW) model mainly consider the spatial arrangement of an image, ignoring the rich textural information in land-use remote-sensing images. Hence, this article presents a 2-D wavelet decomposition (WD)-based BOVW model for land-use scene classification, since the 2-D wavelet decomposition method does well not only in textural feature extraction, but also in the multi-resolution representation of an image, which is favourable for the use of both spatial arrangement and textural information in land-use images. The proposed method exploits the textural structures of an image with colour information transformed into greyscale. Moreover, it works first by decomposing the greyscale image into different sub-images using 2-D discrete wavelet transform (DWT) and then by extracting local features of the greyscale image and all the decomposed images with dense regions in which a given image is evenly sampled by a regular grid with a specified grid space. After that, the method generates the corresponding visual vocabularies and computes histograms of visual word occurrences of local features found in each former image. Specifically, the soft-assignment or multi-assignment (MA) technique is employed, accounting for the impact of clustering on visual vocabulary creation that two similar image patches may be clustered into different clusters when increasing the size of visual vocabulary. The proposed method is evaluated on a ground truth image dataset of 21 land-use classes manually extracted from high-resolution remote-sensing images. Experimental results demonstrate that the proposed method significantly outperforms previous methods, such as the traditional BOVW model, the spatial pyramid representation-based BOVW method, the multi-resolution representation-based BOVW method, and so on, and even exceeds the best result obtained from the creator of the land-use dataset. Therefore, the proposed approach is very suitable for land-use scene classification tasks.

Land-cover classification especially global mapping has become a new trend in recent years. Traditional convolutional neural network (CNN) methods for land-cover classification are usually patch based and have problems of high computation cost and low efficiency, which hinder their wide applications when timely and accurately mapping land covers. Fortunately, methods based on the fully convolutional network (FCN) have achieved state-of-the-art performance on the semantic segmentation task, which provides a new possibility for efficient land-cover classification. Many works have been done for land-cover classification but are almost focused on very high-resolution remote-sensing images and few research works are implemented on medium-resolution images. In this paper, six representative state-of-the-art segmentation models including ‘U’-shaped network (U-Net), fully convolutional DenseNet (FC-DenseNet), full-resolution residual network (FRRN), bilateral segmentation network (BiSeNet), DeepLab version 3 plus (DeepLabV3+), and pyramid scene parsing network (PSPNet) are selected to compare their performances on the land-cover classification of Land Remote-Sensing Satellite System)-5 satellite remote-sensing images. Based on the analysis of their performances, an improved model named atrous spatial pyramid pooling U-Net (ASPP-U-Net) is proposed for classification. Methods including support vector machine, patch-based CNN, and U-Net are also selected for comparison with the proposed model. Furthermore, to overcome the insufficiency of reference data when training deep models, an integration strategy based on two existing global land-cover products finer resolution observation and monitoring of global land cover of 2010 and global land-cover mapping at 30 m resolution is designed to produce reference data. Experimental results show that the encoder–decoder architecture especially U-Net is the most competitive network and is highly recommended for mapping land covers of medium-resolution images. The proposed ASPP-U-Net outperforms other compared methods not only in the classification accuracy but also in the inference time efficiency. In addition, it is advisable to use existing global land-cover products to produce reference data for segmentation models when the labelled datasets are insufficient.

Land cover classification is one of the most fundamental tasks in the field of remote sensing. In recent years, fully supervised fully convolutional network (FCN)-based semantic segmentation models have achieved state-of-the-art performance in the semantic segmentation task. However, creating pixel-level annotations is prohibitively expensive and laborious, especially when dealing with remote sensing images. Weakly supervised learning methods from weakly labeled annotations can overcome this difficulty to some extent and achieve impressive segmentation results, but results are limited in accuracy. Inspired by point supervision and the traditional segmentation method of seeded region growing (SRG) algorithm, a weakly towards strongly (WTS) supervised learning framework is proposed in this study for remote sensing land cover classification to handle the absence of well-labeled and abundant pixel-level annotations when using segmentation models. In this framework, only several points with true class labels are required as the training set, which are much less expensive to acquire compared with pixel-level annotations through field survey or visual interpretation using high-resolution images. Firstly, they are used to train a Support Vector Machine (SVM) classifier. Once fully trained, the SVM is used to generate the initial seeded pixel-level training set, in which only the pixels with high confidence are assigned with class labels whereas others are unlabeled. They are used to weakly train the segmentation model. Then, the seeded region growing module and fully connected Conditional Random Fields (CRFs) are used to iteratively update the seeded pixel-level training set for progressively increasing pixel-level supervision of the segmentation model. Sentinel-2 remote sensing images are used to validate the proposed framework, and SVM is selected for comparison. In addition, FROM-GLC10 global land cover map is used as training reference to directly train the segmentation model. Experimental results show that the proposed framework outperforms other methods and can be highly recommended for land cover classification tasks when the pixel-level labeled datasets are insufficient by using segmentation models.