Multimodal change detection (MCD) has garnered significant interest due to its capacity to address a variety of emergencies in a timely and effective manner. However, discrepancies in sensors and imaging techniques often hinder the direct comparison of heterogeneous remote sensing images (HRSIs), making it difficult to extract change information. To overcome this challenge, we propose a novel superpixel-based AutoEncoder Kolmogorov-Arnold Network (AEKAN) for unsupervised MCD. The primary objective of AEKAN is to excavate the latent commonality features between HRSIs. Notably, commonality features in unchanged regions are generally more pronounced than those in changed regions, which can be leveraged to assess change magnitude. To achieve this, the proposed method utilizes the Kolmogorov-Arnold Network (KAN), renowned for its capability to model data distributions, to extract these commonality features between HRSIs. Concretely, the proposed AEKAN consists of a Siamese KAN encoder and dual KAN decoders. The Siamese encoder aims to map HRSIs and extract latent commonality features, while the dual decoders reconstruct original bi-temporal images from these features. Additionally, we incorporate a hierarchical commonality loss function within the Siamese encoder to train AEKAN. This loss function is designed to intentionally guide the network in capturing commonality features by minimizing the discrepancies in features extracted from HRSIs at each layer of the Siamese encoder. The extracted commonality features are then adopted to quantify the change magnitude between images through mean square error. Extensive experiments on five MCD datasets demonstrate that the proposed AEKAN outperforms existing methods. The source code is available at https://github.com/TongfeiLiu/AEKAN-for-MCD.
The hybrid architecture of convolutional neural networks (CNNs) and Transformer are very popular for medical image segmentation. However, it suffers from two challenges. First, although a CNNs branch can capture the local image features using vanilla convolution, it cannot achieve adaptive feature learning. Second, although a Transformer branch can capture the global features, it ignores the channel and cross-dimensional self-attention, resulting in a low segmentation accuracy on complex-content images. To address these challenges, we propose a novel hybrid architecture of convolutional neural networks hand in hand with vision Transformers (CiT-Net) for medical image segmentation. Our network has two advantages. First, we design a dynamic deformable convolution and apply it to the CNNs branch, which overcomes the weak feature extraction ability due to fixed-size convolution kernels and the stiff design of sharing kernel parameters among different inputs. Second, we design a shifted-window adaptive complementary attention module and a compact convolutional projection. We apply them to the Transformer branch to learn the cross-dimensional long-term dependency for medical images. Experimental results show that our CiT-Net provides better medical image segmentation results than popular SOTA methods. Besides, our CiT-Net requires lower parameters and less computational costs and does not rely on pre-training. The code is publicly available at https://github.com/SR0920/CiT-Net.
The current deep convolutional neural networks for very-high-resolution (VHR) remote-sensing image land-cover classification often suffer from two challenges. First, the feature maps extracted by network encoders based on vanilla convolution usually contain a lot of redundant information, which easily causes misclassification of land cover. Moreover, these encoders usually require a large number of parameters and high computational costs. Second, as remote-sensing images are complex and contain many objects with large-scale variances, it is difficult to use the popular feature fusion modules to improve the representation ability of networks. To address the above issues, we propose a dynamic convolution self-attention network (DCSA-Net) for VHR remote-sensing image land-cover classification. The proposed network has two advantages. On one hand, we designed a lightweight dynamic convolution module (LDCM) by using dynamic convolution and a self-attention mechanism. This module can extract more useful image features than vanilla convolution, avoiding the negative effect of useless feature maps on land-cover classification. On the other hand, we designed a context information aggregation module (CIAM) with a ladder structure to enlarge the receptive field. This module can aggregate multi-scale contexture information from feature maps with different resolutions using a dense connection. Experiment results show that the proposed DCSA-Net is superior to state-of-the-art networks due to higher accuracy of land-cover classification, fewer parameters, and lower computational cost. The source code is made public available.
1 January 2021 in International Journal of Agricultural and Biological Engineering
Underwater images suffer from light absorption and scattering, impairs their visibility and applications. Existing underwater image restoration (UIR) methods based on generative models struggle are difficult to adapt to the complex and dynamic underwater environments characterized by illumination interference, low-light conditions, and non-uniform turbidity. To address these issues, we propose Water-CDM, a novel Adaptive Double-Branch Fusion Conditional Diffusion Model for underwater image restoration. Specifically, an adaptive double-branch fusion conditional diffusion model is presented utilizing a U-shaped full-attention network and Guided Multi-Scale Retinex with Brightness Correction (GMSRBC) to restore the challenging regions within underwater images. More precisely, to correct color casts and enhance the sharpness of underwater images, a U-shaped full-attention network incorporating Attention Blocks is designed for noise estimation during the reverse process of the conditional diffusion model. Concurrently, to mitigate overexposure during the enhancement of low-light underwater images under illumination interference, the GMSRBC method, featuring an Adaptive Brightness Correction Module, is proposed to efficiently adjust the brightness of underwater images. Experimental results demonstrate that the proposed Water-CDM significantly improves the quality of underwater images in challenging scenarios. Encouragingly, our proposed Water-CDM yields superior restoration outcomes compared to current state-of-the-art methods on three challenging publicly available datasets. Our codes will be released at: https://github.com/HKandWJJ/Water-CDM.
Target tracking and imaging through scattering media are challenges that must be overcome for important applications in various fields. The fundamental problem is the randomly diffused light in scattering media that prevents the formation of diffraction-limited images. In recent years, speckle correlation has emerged as a powerful approach for restoring targets through scattering media and tracking targets with high scattering. However, this method fails to achieve high-accuracy target tracking in multidimensional motion. Furthermore, retrieving the deterministic phase of a target in a low-resolution speckle pattern is limited. Here, a method based on speckle-difference-combined bispectrum analysis (SDCBA) is presented to simultaneously track and image targets. The proposed SDCBA requires no high-resolution pattern, randomly assigned initial values, nor numerous iterations. The target is tracked simultaneously in multidimension by using speckle difference autocorrelation and restored with the deterministic phase via bispectrum analysis of the low-resolution speckle pattern. This work carries out simulations and experiments to demonstrate simultaneous multidimensional target tracking and imaging in low-resolution speckle pattern through scattering media via the SDCBA. This work will benefit various fields, including biomedical applications, materials science, and military security.