When segmenting massive amounts of remote sensing images collected from different satellites or geographic locations (cities), the pre-trained deep learning models cannot always output satisfactory predictions. To deal with this issue, domain adaptation has been widely utilized to enhance the generalization abilities of the segmentation models. Most of the existing domain adaptation methods, which based on image-to-image translation, firstly transfer the source images to the pseudo-target images, adapt the classifier from the source domain to the target domain. However, these unidirectional methods suffer from the following two limitations: (1) they do not consider the inverse procedure and they cannot fully take advantage of the information from the other domain, which is also beneficial, as confirmed by our experiments; (2) these methods may fail in the cases where transferring the source images to the pseudo-target images is difficult. In this paper, in order to solve these problems, we propose a novel framework BiFDANet for unsupervised bidirectional domain adaptation in the semantic segmentation of remote sensing images. It optimizes the segmentation models in two opposite directions. In the source-to-target direction, BiFDANet learns to transfer the source images to the pseudo-target images and adapts the classifier to the target domain. In the opposite direction, BiFDANet transfers the target images to the pseudo-source images and optimizes the source classifier. At test stage, we make the best of the source classifier and the target classifier, which complement each other with a simple linear combination method, further improving the performance of our BiFDANet. Furthermore, we propose a new bidirectional semantic consistency loss for our BiFDANet to maintain the semantic consistency during the bidirectional image-to-image translation process. The experiments on two datasets including satellite images and aerial images demonstrate the superiority of our method against existing unidirectional methods.
26 December 2022 in IEEE Transactions on Geoscience and Remote Sensing
Unsupervised domain adaptation (UDA) has become an important technique for cross-domain semantic segmentation (SS) in remote sensing community and obtained remarkable results. However, when transferring from high-resolution (HR) remote sensing images to low-resolution (LR) images, the existing UDA frameworks always fail to segment the LR target images, especially for small objects (e.g., cars), due to the severe spatial resolution shift problem. In this paper, to improve the segmentation ability of UDA models for LR target images and small objects, we propose a novel multi-task domain adaptation network (DASRSNet) for SS of remote sensing images with the aid of super-resolution (SR). The proposed DASRSNet contains domain adaptation for semantic segmentation (DASS) branch, domain adaptation for super-resolution (DASR) branch and feature affinity (FA) module. Specifically, the DASS and DASR branches share the same encoder to extract the domain-invariant features for the target and source domains, and these two branches utilize different decoders and discriminators to conduct cross-domain SS task and SR task, which align the domain shift in output space and image space respectively. Finally, FA module, which involves the proposed FA loss, is applied to enhance the affinity of SS features and SR features for both source and target domains. The experimental results on the cross-city aerial datasets demonstrate the effectiveness and superiority of our DASRSNet against the recent UDA models.
In China, due to the fast growing of urbanization and industrialization, many water-bodies have been affected to show strong evidence of Black and Odorous (BO) properties, which cause a huge impact on the people’s daily life and destroy the ecosystem around the water-bodes. Therefore, it is very important to monitor the BO condition in the urban area. In order to monitor the black and odorous water (BOW) and protect the environment in the urban area, the government has issued a regulation for urban black odorous water monitoring named as “Action Plan for Prevention and Control of Water Pollution” in 2015. However, there does not have an effective method to monitor the BOW from the macro level until the employment of remote sensing technology. The remote sensing technology uses the optical characteristics of the water body, and the spectral properties of the reflection and absorption of organic matter in the water to inverse the BO level. Due to faster progress of remote sensing technology, the monitoring of BO by using remotes sensing data become more and more accurate and efficient. The Chinese High-resolution Earth Observation System (CHEOS) is one the major project of China. It consists of five sub-system, the space-bornee observation system, near space observation system, aerial observation system, data management system and application system. For space-bornee observation system, it has seven series of observation satellites in a constellation which are named as GF1-GF7. GF1 and its counterpart satellites GF1B/C/D have optical payloads with 2m spatial resolution in Panchromatic mode (Pan) and is able to working together as sub-constellation to improve the time resolution of GF1. GF2 is a sub-meter optical observation satellite. GF3 is a C-band multi-mode Synthetic Aperture Radar (SAR) satellite. GF4 is a geostationary earth orbit optical satellite for the purpose of high time resolution imaging with the characteristics of 3 images per minutes. GF5 is a hyper-spectrum optical satellite with five other different payload for atmosphere and land monitoring. GF6 is specially designed for the application of smart agriculture with its four added spectral bands compared with GF1. GF7 is 3D sub-meter mapping satellite with three payloads. In order to demonstrate the inversion of BO level by using GF remote sensing data, the Luoshe town is selected with the characteristics of massive water body network and fast growing in urbanization. In this paper, the general payloads information of GF series satellites have been introduced. The demonstration of BOW inversion by using GF series remote sensing data in the area of Luoshe is performed over a period form 2019 to 2020. The results demonstrate a clear pattern of BO changing along with the changing of seasonal temperature and the water flow dynamic. In conclusion, the BOW monitoring method by using GF series remote sensing date is highly effective, specially in a large scale with a time frame can be achieved by weekly or daily with the fusion of multi-satellite data.
High-resolution remote-sensing images contain dense object layouts and fine spatial details, but their spatial extent makes direct vision–language interpretation difficult. Patch retrieval offers a practical alternative by returning local regions as candidate evidence for downstream interpretation. However, independent semantic scores can leave relevant patches at isolated ranks or return lists with substantial irrelevant content. We propose Prompt-Stable Reciprocal Inter-scale Visual Evidence Reranking (PRIVER), a post-retrieval method for ranking patches within one image using controlled object-category queries. It aggregates four predefined prompt templates and combines semantic scores with geometric support, principally from a degree-normalized inter-scale graph. No encoder weights are updated, and retrieval and reranking do not access object annotations. Parameters are selected on annotated DOTA-v1.5 development data and held fixed. With relevance defined by annotated object coverage, Precision@10 on 458 DOTA-v1.5 internal-test images increases from 0.567 to 0.625 (absolute gain 0.058; 95% source-image cluster bootstrap interval [0.052, 0.065]). Normalized discounted cumulative gain at rank 10 (nDCG@10) increases from 0.647 to 0.730 and distinct-object coverage at rank 10 from 0.773 to 0.873. On a disjoint 526-image SODA-A subset, the corresponding metrics increase from 0.414 to 0.502, 0.445 to 0.550, and 0.467 to 0.681. Coverage-based gains persist with GeoRSCLIP and RemoteCLIP and under altered tiling policies on fixed image subsets. Inter-scale support is the principal source of improvement. These results support training-free, coverage-defined patch prioritization for within-image visual evidence retrieval.