In recent years, supervised learning, represented by deep learning, has shown good performance in remote sensing image scene classification with its powerful feature learning ability. However, this method requires large-scale and high-quality handcrafted labeled datasets, which leads to a high cost of obtaining annotated samples. Self-supervised learning can alleviate this problem by using unlabeled data to learn the image’s feature representation and then migrate to the downstream task. In this study, we use an encoder–decoder structure to construct a self-supervised learning architecture. In the encoding stage, the image mask is used to discard some of the image patches randomly, and the image’s feature representation can be learned from the remaining image patches. In the decoding stage, the lightweight decoder is used to recover the pixels of the original image patches according to the features learned in the encoding stage. We constructed a large-scale unlabeled training set using several public scene classification datasets and Gaofen-2 satellite data to train the self-supervised learning model. In the downstream task, we use the encoder structure with the masked image patches that have been removed as the backbone network of the scene classification task. Then, we fine-tune the pre-trained weights of self-supervised learning in the encoding stage on two open datasets with complex scene categories. The datasets include NWPU-RESISC45 and AID. Compared with other mainstream supervised learning methods and self-supervised learning methods, our proposed method has better performance than the most state-of-the-art methods in the task of remote sensing image scene classification.

Deep neural networks driven by large amounts of annotated samples have been successfully applied in remote sensing scene classification (RSSC). On most common public datasets, the accuracies of scene recognition tasks have been close to saturation. A learning paradigm that specifically tackles this issue has emerged, i.e., few-shot learning (FSL). FSL methods for natural image recognition are developing rapidly. Recently, these methods have been widely used in RSSC. Compared with natural images, the problems of intra-class differences and inter-class similarities in RSSC are more serious, which hinders the further development of FSL methods for RSSC. To address this issue, we propose a multi-manifold metric learning framework for RSSC with FSL. Specifically, we use a lightweight convolutional neural network as a feature extraction block. To further enhance the representation ability of features, we embed the feature map into two heterogeneous and complementary Riemannian manifold geometric structures, i.e. Grassmannian manifold and symmetric positive definite (SPD) manifold. Then, to facilitate the measurement, we introduce the Riemann kernel function to embed two heterogeneous manifold structures into the high-dimensional Hilbert space for fusion. Finally, we design a learnable distance metric scheme that can be optimized according to the divergence of inter-class pairs and intra-class pairs, thereby reducing the impact of large intra-class differences and high inter-class similarity on scene recognition. We verify the effectiveness of MMML on three commonly used datasets. The results show that the accuracy of our proposed method is from 1.21% to 3.12% higher than the state-of-the-art methods.

In urban scenes, there are man-made ground objects with complex structures and significant height differences, which leads to challenges in generating large-scale true digital orthophoto maps (TDOMs) of cities. However, traditional TDOM generation methods rely on precise prior geometric information, and this reliance makes it difficult to adapt to such complex environments. In this work, we propose TDOM-NeRF, a novel neural radiance field (NeRF) based method for large-scale urban TDOM generation. Without relying on additional prior information, our method utilizes multiview unmanned aerial vehicle (UAV) images as input and implicitly expresses the scene using hash grid features and multilayer perceptrons. By performing orthogonal volume rendering on the reconstruction results of the scene, we effectively address the problem of uneven scales in the synthetic views when generating TDOM. In actual training, we adopt a scene-block training strategy to extend our method to TDOM generation of large-scale scenes while achieving high-fidelity scene reconstruction. In addition, we regularize the model during training to eliminate erroneous “floating objects” in scene reconstruction, making our method applicable to low-overlap UAV datasets. Experimental results demonstrate that our method generates TDOMs with comparable geometric accuracy and eliminates issues such as ghosting, misalignment, and distortion in challenging areas, compared to commercial photogrammetry software like Metashape and Pix4DMapper. Furthermore, our method exhibits superior novel view synthesis performance compared to advanced NeRF methods like Ortho-NeRF. Our pipeline offers a flexible configuration of scene blocking, enabling a balance between performance and efficiency, which makes it particularly suitable for large-scale scene reconstruction.

Scene classification is of great significance to understand the semantics and extract the information of high spatial resolution remote sensing image. The bag-of-visual-words( BOVW) model is an effective method to understand the semantic content of images. It is widely used in scene classification of remote sensing image.The existed BOVW models usually use single feature or multiple features (such as spectrum, texture, and shape) to describe visual words. However, when considering multiple low-level feature fusion strategies, most methods only combine them by simple accumulation or concatenation which can not fully learn the relationship between different features.In this letter, a visual bag-of-words scene classifier based on regional covariance (RCOVBOVW) is proposed. This method can naturally fuse multiple related features, and the covariance calculation itself has the filtering ability, which can also reduce the dimension of the features and have high efficiency. Experiments have been conducted on two public and challenging datasets (UC Merced and NWPU-RESISC45), and the results show that our proposed method outperforms the most state-of-the-art methods of remote sensing image scene classification.

Few-shot learning (FSL) is a method that does not require a large number of labeled samples and has ability to identify unseen samples. Recently, many excellent FSL methods have been developed for remote sensing scene classification (RSSC). However, for complex remote sensing scenes, the following problems have not been well addressed: 1) the order of local features and the spatial correlation between features haven’t been taken into consideration, 2) the current metric methods are mainly based on Euclidean space (ES), and discriminative ability is limited. Hence, we propose a framework for RSSC with FSL by combining Hilbert feature sequence and Lie group metric space (HFS-LGMS). Specifically, we adopt a lightweight 4-layers convolutional neural network (CNN) as a feature extractor. For the last feature map, we regard each pixel as a multi-dimensional local feature. Then we organize all the features into a feature sequence along the Hilbert curve. To represent the correlation among features, we introduce the autoregressive moving average model (AMAM) to model the observed features and their spatial states. Since the parameters of the model are matrices, which are directly processed as vectors will affect performance. Therefore, we transform the matrix parameter space into the Lie group manifold space (LGMS), and introduce the Lie group intrinsic mean (LGIM) to represent the prototype. The distance between the Lie groups is measured by the geodesic distance. The similarity is measured on Lie group space. Finally, the effectiveness of HFS-LGMS is demonstrated on three public datasets.