18 January 2022 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Due to technology and cost limitations, it is challenging to obtain high temporal and spatial resolution images from a single satellite spectrometer, which significantly limits the specific application of such remote sensing images in earth science. To solve the problem that the existing algorithms cannot effectively balance the spatial detail preservation and spectral change reconstruction, a pseudo-Siamese deep convolutional neural network (PDCNN) for spatiotemporal fusion is proposed in this article. The method proposes a pseudo-Siamese network framework model for fusion. This framework has two independent and equal feature extraction streams, but the weights are not shared. The two feature extraction streams process the image information at the previous and later moments and reconstruct the fine image of the corresponding time to fully extract the image information at different times. In the feature extraction stream, the multiscale mechanism and dilated convolution of flexible perception are designed, which can flexibly obtain feature image information and improve the model reconstruction accuracy. In addition, an attention mechanism is introduced to improve the weight of the crucial information for the remote sensing images. Adding a residual connection enhances the reuse of the initial feature information in shallow networks and reduces the loss of feature information in deep networks. Finally, the fine images obtained from the two feature extraction streams are weighted and fused to obtain the final predicted image. The subjective and objective results demonstrate that the PDCNN can effectively reconstruct the fusion image with higher quality.
16 September 2021 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Remote sensing satellite images with high temporal and high spatial resolution play a critical role in earth science applications. However, it is difficult for a single satellite to obtain such images due to technical and cost constraints. Therefore, spatiotemporal image fusion based on deep learning has received extensive attention in recent years. This article proposes a multicooperative deep convolutional neural network (MCDNet) for spatiotemporal satellite image fusion. This method is a new multinetwork model in which multiple networks work together to reconstruct the predicted image. The multinetwork model consists of a super-resolution network, a difference reconstruction network, and a collaborative training network. First, the super-resolution network uses the combination of a novel multiscale mechanism and dilated convolutions to make full use of the spectral information of the coarse image and upgrade it to a transitional image that matches the fine image. The difference reconstruction network uses structural relevance to complete the reconstruction of the fine difference image. The collaborative training network extracts the hidden information from the fine image and uses the time relevance to restrict the training of the difference reconstruction network. Finally, the fine difference image and the known fine image are combined to complete the image fusion. The new compound loss function can help multinetwork models better complete cooperative training. Through experiments on two datasets and comparison with existing fusion algorithms, the subjective and objective results prove that MCDNet can effectively reconstruct higher-quality prediction images.