The existing publicly available datasets with pixel-level labels contain limited categories, and it is difficult to generalize to the real world containing thousands of categories. In this paper, we propose an approach to generate object masks with detailed pixel-level structures/boundaries automatically to enable semantic image segmentation of thousands of targets in the real world without manually labelling. A Guided Filter Network (GFN) is first developed to learn the segmentation knowledge from an existed dataset, and such GFN then transfers the learned segmentation knowledge to generate initial coarse object masks for the target images. These coarse object masks are treated as pseudo labels to self-optimize the GFN iteratively in the target images. Our experiments on six image sets have demonstrated that our proposed approach can generate object masks with detailed pixel-level structures/boundaries, whose quality is comparable to the manually-labelled ones. Our proposed approach also achieves better performance on semantic image segmentation than most existing weakly-supervised, semi-supervised, and domain adaptation approaches under the same experimental conditions.
16 May 2023 in IEEE Transactions on Geoscience and Remote Sensing
Cloud detection is one of the key technologies in the field of remote sensing. Although extensive deep learning-based cloud detection methods achieve good performance, their detection results in confusing areas such as cloud boundaries and thin clouds are often not satisfactory due to the potential inter-class similarity and intra-class inconsistency of objects. To this end, we propose a Boundary-Aware Bilateral Fusion network (BABFNet), which effectively enhances cloud detection in confusing areas by introducing a boundary prediction branch as an auxiliary. To avoid the loss of details, the boundary prediction branch is designed to run at full resolution with a shallow architecture, while some Semantic Enhancement Modules (SEMs) are used to supplement high-level semantic information by introducing multi-level encoder features of the cloud detection branch. This feature sharing in turn drives the cloud detection branch to focus more on cloud boundaries during training. At the end of the network, a Bilateral Fusion Module (BFM) is added for information complementarity between features from these two branches. The features from the cloud detection branch provide multi-scale features to the boundary prediction branch for more accurate boundary prediction, while the features from the boundary prediction branch further serve as prior knowledge to help the cloud detection branch aggregate contextual information. To verify the effectiveness of the proposed method, we select four different networks as cloud detection branches and conduct comparative experiments on two public datasets, GF-1 WFV and MODIS. The experimental results show that the proposed method significantly enhances cloud detection in confusing areas.
Existing image super-resolution methods have made remarkable advancements in enhancing the visual quality of real-world images. However, when it comes to restoring Chinese paintings, these methods encounter unique challenges. This is primarily due to the difficulty in preserving intricate non-realistic details and capturing comple semantic information with high dimensionality. Moreover, the preservation of the original artwork’s distinct style and subtle artistic nuances further amplifies this complexity. To address these challenges and effectively restore traditional Chinese paintings, we propose a Convolutional Super-Resolution Generative Adversarial Network for Chinese landscape painting super-resolution, termed ConvSRGAN. We employ Enhanced Adaptive Residual Module to delve deeply into multi-scale feature extraction in images, incorporating an Enhanced High-Frequency Retention Module that leverages an Adaptive Deep Convolution Block to capture fine-grained high-frequency details across multiple levels. By combining the Multi-Scale Structural Similarity loss with conventional losses, our ConvSRGAN ensures that the model produces outputs with improved fidelity to the original image’s texture and structure. Experimental validation demonstrates significant qualitative and quantitative results when processing traditional paintings and murals datasets, particularly excelling in high-definition reconstruction tasks for landscape paintings. The reconstruction effect showcases enhanced visual fidelity and liveliness, thus affirming the effectiveness and applicability of our approach in cultural heritage preservation and restoration.
With the continuous improvement of high-resolution remote-sensing image-acquisition technologies, image quality and resolution are constantly improved, which greatly promotes the development of object detection in large-scale remote sensing images. However, due to the memory constraints of Graphics Processing Units (GPUs), directly inputting large-scale remotesensing images into detectors poses significant challenges. Downsampling these images to sizes suitable for detectors often leads to the loss of critical details. Current mainstream object detection methods for large-scale images typically divide large images into patches and detect the objects of interest in all patches. These methods allocate computational resources uniformly on each patch, regardless of the presence of objects or variations in their density and size. This not only disrupts the delicate balance between accuracy and speed but also results in inefficient utilization of computational resources. To address these challenges, we propose a Situation-Aware Network (SANet) for object detection in large-scale remote sensing images. SANet introduces a Situation-Aware (SA) module designed to filter out invalid information in large-scale images, thereby reducing computational complexity. Moreover, it integrates a RegionAdaptive (RA) module, which dynamically adjusts the level of granularity for object regions and allocates them to appropriate detectors, thus minimizing resource wastage. Our method has been tested on challenging datasets such as DOTA-v1.0, DOTAv1.5, DOTA-v2.0, and TT100K, demonstrating consistent advantages in both speed and accuracy compared to existing object detection methodologies.