4 March 2024 in IEEE Transactions on Geoscience and Remote Sensing
Multi-modal remote sensing images registration ensures the consistency of the spatial positions for different images. It can provide the accurate geographic information and supports the fusion of multi-source data for geospatial analyses and applications. Rigid registration method shows high performance in dealing with large-scale deformation, but it is difficult to achieve high-precision image registration. In contrast, non-rigid registration method is suitable for processing local differences, but cannot effectively deal with large-scale deformation differences. Therefore, the combination of rigid and non-rigid registration methods becomes a necessary strategy to address such issues. In this paper, we propose a novel ADRNet method for multi-modal remote sensing images registration. The proposed ADRNet method contains three main modules: affine registration module, deformable registration module, and spatial transformer module that integrates the affine and deformable transformation parameters to obtain the final aligned images. Meanwhile, we design a new feature enhancement module and an attention module with dilated convolutions which have different dilation rates, which are used to alleviate the limitations imposed by receptive fields in the convolution operation. Moreover, we propose a specific symmetric loss function to optimize the whole network from the perspective of inverse consistency. To assess the efficiency and performance of the network, we extend the experimental data, ranging from cross-modal images in a conventional viewpoint to cross-modal images in a remote sensing viewpoint. The experimental results show that our method exhibits excellent performance for the images with different viewpoints and deformation scales. The relevant code will be released at: https://github.com/Ahuer-Lei/ADRNet.
1 September 2022 in International Journal of Computer Vision
26 August 2022 in IEEE Transactions on Circuits and Systems for Video Technology
Contouring artifacts usually appear in large and smooth flat areas, which are caused by many widely used processes such as bit-depth expansion, compression, image sharpening and contrast enhancement. Unfortunately, recent decontouring methods were mainly designed for specific and non-blind degradations, which significantly reduces the generalization ability of these methods when applied to complex and various real-world false contours. Therefore, this paper explores the blind decontouring problem by proposing a blind decontouring network (BDCN). Instead of directly training a decontouring network with mixed degradations, the proposed model consists of two independent modules, i.e., a flat region detection module (FDM) and a decontouring module (DCM). The FDM is designed to extract flat region masks robust to various false contours, which can preserve texture details from global smoothing. Then, the task of DCM becomes simply smoothing different contouring artifacts. Both the FDM and DCM are designed with a lightweight architecture and reparameterization strategy. Experimental results on both synthetic and real-world contouring artifacts demonstrate the effectiveness and generalization of the proposed method.
21 October 2025 in Machine Intelligence Research
30 November 2021 in IEEE Transactions on Neural Networks and Learning Systems
Although the cartoon industry has developed rapidly in recent years, few studies pay special attention to cartoon image quality assessment (IQA). Unfortunately, applying blind natural IQA algorithms directly to cartoons often leads to inconsistent results with subjective visual perception. Hence, this brief proposes a blind cartoon IQA method based on convolutional neural networks (CNNs). Note that training a robust CNN depends on manually labeled training sets. However, for a large number of cartoon images, it is very time-consuming and costly to manually generate enough mean opinion scores (MOSs). Therefore, this brief first proposes a full reference (FR) cartoon IQA metric based on cartoon-texture decomposition and then uses the estimated FR index to guide the no-reference IQA network. Moreover, in order to improve the robustness of the proposed network, a large-scale dataset is established in the training stage, and a stochastic degradation strategy is presented, which randomly implements different degradations with random parameters. Experimental results on both synthetic and real-world cartoon image datasets demonstrate the effectiveness and robustness of the proposed method.
3 June 2025 in IEEE Transactions on Geoscience and Remote Sensing
Multimodal remote sensing image registration ensures that images from different sensors or modalities are spatial and informational consistent for effective comparison and analysis. However, due to the nonlinear modality gaps that exist between images, it is difficult to focus solely on spatial positional differences while ignoring the modality gaps. In this article, to address this issue, we propose a new framework for multimodal registration network, named MMRNet. The proposed framework comprises the following main aspects. First, a novel self-supervised positional misalignment estimator (PME) is designed for multimodal image registration. PME can efficiently overcome the modality gaps and learn the positional differences between multimodal images more reliably, optimizing the registration loss by minimizing the positional differences directly. Then, a new paradigm of modality translation, termed modality perception module (MPM), is introduced to effectively learn modality gaps and perform modality translation in the case of positional misalignment. Finally, we further design the modality perception guidance loss to supervise the modality translation task, which can encourage the fidelity of the generated pseudo-modality images. Our registration network integrates both rigid registration model and nonrigid registration model. The experimental results demonstrate that the proposed registration framework can obtain obviously superior performance in both rigid and nonrigid image registration tasks on optical-synthetic aperture radar (SAR) data, optical-map data, and optical-infrared data. The code and relevant dataset will be made publicly available at https://github.com/Ahuer-Lei/MMRNet.