15 October 2021 in IEEE Transactions on Geoscience and Remote Sensing
The low resolution of remote sensing images often limits the land cover classification (LCC) performance. Super resolution (SR) can improve the image resolution, while greatly increasing the computational burden for the LCC due to the larger size of the input image. In this article, the SR-guided deep network (SRGDN) framework is proposed, which can generate meaningful structures from higher resolution images to improve the LCC performance without consuming more computational costs. In general, the SRGDN consists of two branches (i.e., SR branch and LCC branch) and a guidance module. The SR branch aims to increase the resolution of remote sensing images. Since high- and low-resolution image pairs cannot be directly provided by imaging sensors to train the SR branch, we introduce a self-supervised generative adversarial network (GAN) to estimate the downsampling kernel that can produce these image pairs. The LCC branch adopts the high-resolution network (HRNet) to retain as much resolution information with a few downsampling operations as possible. The guidance module teaches the LCC branch to learn the high-resolution information from the SR branch without the utilization of the higher-resolution images as the inputs. Furthermore, the guidance module introduces spatial pyramid pooling (SPP) to match the feature maps of different sizes in the two branches. In the testing stage, the guidance module and SR branch can be removed, and therefore do not create additional computational costs. Experimental results on three real datasets demonstrate the superiority of the proposed method over several well-known LCC approaches.
15 November 2024 in Applied Acoustics
11 April 2024 in IEEE Transactions on Geoscience and Remote Sensing
The spectral super-resolution (SpeSR) from multispectral images (MSIs) to hyperspectral images (HSIs) can bring rich spectral information. The deep learning-based methods have demonstrated their powerful ability for the SpeSR task, which requires the paired HSI/MSI to train the model. However, HSIs and MSIs are always obtained at different times and under different imaging conditions, covering different areas. To address this issue, in this article, a framework named BliEstGAN based on the generative adversarial network (GAN) is proposed to estimate the spectral resolution degradation between unpaired HSIs and MSIs that can be used for the blind SpeSR. Specifically, each MSI imaging sensor has its own unique spectral sampling process, which can be modeled as a spectral degradation from its paired HSI. Different spectral degradations can be discriminated by the deep model. Therefore, the generator of the GAN is used to estimate the spectral degradation from HSIs to MSIs, and the discriminator of the GAN is adopted to distinguish whether the estimated and real spectral degradation is similar. The MSIs and HSIs are easy to discriminate against by the discriminator because their spatial resolutions are different. Furthermore, according to the imaging sensor mechanism, some special regularization terms are designed for the generator to guarantee its correct convergence. Finally, the estimated spectral resolution degradation can be adopted to generate HSI/MSI pairs for the supervised learning-based SpeSR methods. Experimental results demonstrate the effectiveness of the proposed method.
16 January 2025 in Optics Express
Optical information synthesis, which fuses LiDAR and optical cameras, has the potential for highly detailed 3D representations. However, due to the disparity of information density between point clouds and images, conventional matching methods based on points often lose significant information. To address this issue, we propose a regional matching method to bridge the differences in information density between point clouds and images. In detail, fine semantic regions are extracted from images by analyzing their gradients. Simultaneously, point clouds are transformed into meshes, where each facet corresponds to a coarse semantic region. Extrinsic matrices are used to unify the point cloud coordinate system with the image coordinate system. The mesh is further subdivided based on the guidance of image texture information to create regional matching units. Within each matching unit, the information density of the point cloud and the image is carefully balanced at a semantic level. The texture features of the image are well preserved in the transformed mesh structure. Consequently, the proposed texture synthesis method significantly enhances the overall quality and realism of the 3D imaging.
6 February 2025 in Optics & Laser Technology