Feature pyramid network transformer decoder (FPNFormer) module, which can effectively deal with the strong rotation arbitrary of remote sensing images while improving the expressiveness and robustness of the model. It is a plug-and-play module that can be well transferred to various detection models and significantly improves performance. Specifically, we use the computational method of transformer decoder to deal with the problem that the image has any orientation, and its output weakly depends on the order of the input data. We apply it to the feature fusion stage and design two ways top-down and down-top to fuse features of different scales, which enables the model to have a more vital ability to perceive objects at different scales and angles. Experiments on commonly used benchmarks (DOTA1.0, DOTA1.5, SSDD, and RSDD) demonstrate that the proposed FPNFormer module significantly improves the performance of multiple arbitrary-oriented object detectors, such as 1.99% map improvement of rotated retinanet on DOTA’s cross-validation set. On RSDD datasets, the baseline model using FPNFormer improves the map of large objects by 5.1%. Combined with more competitive models, the proposed method can achieve a 79.39% map on the DOTA1.0 dataset. The code is available at https://github.com/bityangtian/FPNFormer .

Despite promising SLAM research in both vision and robotics communities, which fundamentally sustains the autonomy of intelligent unmanned systems, visual challenges still threaten its robust operation severely. Existing SLAM methods usually focus on specific challenges and solve the problem with sophisticated enhancement or multi-modal fusion. However, they are basically limited to particular scenes with a non-quantitative understanding and awareness of challenges, resulting in a significant performance decline with poor generalization and(or) redundant computation with inflexible mechanisms. To push the frontier of visual SLAM, we propose a fully computational reliable evaluation module called CEMS (Challenge Evaluation Module for SLAM) for general visual perception based on a clear definition and systematic analysis. It decomposes various challenges into several common aspects and evaluates degradation with corresponding indicators. Extensive experiments demonstrate our feasibility and outperformance. The proposed module has a high consistency of 88.298% compared with annotation ground truth, and a strong correlation of 0.879 compared with SLAM tracking performance. Moreover, we show the prototype SLAM based on CEMS with better performance and the first comprehensive CET (Challenge Evaluation Table) for common SLAM datasets (EuRoC, KITTI, etc.) with objective and fair evaluations of various challenges. We make it available online to benefit the community on our website.

Synthetic aperture radar (SAR) has been extensively applied in remote sensing applications. Nevertheless, it is a challenge to process and interpret SAR images. The key to interpreting SAR images lies in transforming them into other forms of remote sensing images to extract valuable hidden remote sensing information. Currently, the conversion of SAR images to optical images produces low-quality results and incomplete spectral information. To address these problems, an end-to-end network model, S2MS-GAN, is proposed for converting SAR images into multispectral images. In this process, to tackle the issues of noise and image generation quality, a TV-BM3D module is introduced into the generator model. Through TV regularization, block-matching, and 3D filtering, these two modules can preserve the edges and reduce the speckle noise in SAR images. In addition, spectral attention is added to improve the spectral features of the generated MS images. Furthermore, we construct a very high-resolution SAR-to-MS image dataset, S2MS-HR, with a spatial resolution of 0.3 m, which is currently the most comprehensive dataset available for high-resolution SAR-to-MS image interpretation. Finally, a series of experiments are conducted on the relevant dataset. Both quantitative and qualitative evaluations demonstrate that our method outperforms several state-of-the-art models in translation performance. The solution effectively facilitates high-quality transitions of SAR images across different types.

Few-shot learning features the capability of generalizing from very few examples. To realize few-shot scene classification of optical remote sensing images, we propose a two-stage framework that first learns a general-purpose representation and then propagates knowledge in a transductive paradigm. Concretely, the first stage jointly learns a semantic class prediction task as well as two auxiliary objectives in a multitask model. Therein, rotation prediction estimates the 2-D transformation of an input, and contrastive prediction aims to pull together the positive pairs while pushing apart the negative pairs. The second stage aims to find an expected prototype having the minimal distance to all samples within the same class. In particular, label propagation (LP) is applied to make joint prediction for both labeled and unlabeled data. Then, the labeled set is expanded by those pseudo-labeled samples, thereby forming a rectified prototype to perform nearest-neighbor classification better. Extensive experiments on standard benchmarks, including Remote sensing image scene classification dataset with 45 classes, published by Northwestern Polytechnical University (NWPU-RESISC45), Aerial Image Dataset (AID), and Remote sensing image scene classification dataset with 19 classes, published by Wuhan University (WHU-RS-19), demonstrate that our method works effectively and achieves the best performance that significantly outperforms many state-of-the-art approaches.

Identifying road surface types is crucial for road maintenance and socio-economic development. Crowdsourced street view data, with global coverage and free access, is a valuable source for this task. However, deep learning models typically require large, manually annotated datasets, which are lacking for road surface classification. To address this, three experts initially labeled 200,000 images from the Mapillary crowdsourced street view dataset, categorizing them into “Paved,” “Unpaved,” and “Unknown” road types through a voting mechanism. Five deep learning models were then trained on this expert-labeled dataset to assess the usefulness of the labels for further analysis. The results show that all models achieved accuracy, precision, recall, and F1 scores above 0.90, with the Swin Transformer model reaching the highest accuracy of 0.924. When applied to six 10 × 10 km sub-regions in Africa, the best model exceeded 0.95 in most performance metrics. These findings demonstrate the potential of this dataset to enhance road surface identification and support infrastructure improvement in Africa.

Frequency domain analysis reveals fundamental image patterns difficult to observe in raw pixel values, while avoiding redundant information in original image processing. Although recent remote sensing foundation models (FMs) have made progress in leveraging spatial and spectral information, they have limitations in fully utilizing frequency characteristics that capture hidden features. Existing FMs that incorporate frequency properties often struggle to maintain connections with the original image content, creating a semantic gap that affects downstream performance. To address these challenges, we propose the All-in-One Spectral-Spatial-Frequency Awareness Foundation Model (Alliance), a framework that effectively integrates information across all three domains. Alliance introduces several key innovations: (1) a progressive frequency decoding mechanism inspired by human visual cognition that minimizes multi-domain information gaps while preserving connections between general image information and frequency characteristics, progressively reconstructing from low to mid to high frequencies to extract patterns difficult to observe in raw pixel values; (2) a triple-domain fusion attention module that separately processes amplitude, phase, and spectral-spatial relationships for comprehensive feature integration; and (3) frequency embedding with frequency-aware Cls token initialization and frequency-specific mask token initialization that achieves fine-grained modeling of different frequency band information. Additionally, to evaluate FMs generalizability, we construct the Yellow River dataset, a large-scale multi-temporal collection that introduces challenging cross-domain tasks and establishes more rigorous standards for FMs assessment. Extensive experiments across six downstream tasks demonstrate Alliance’s superior performance.