18 March 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Precipitation nowcasting refers to the prediction of small-scale precipitation events at minute and kilometer scales within the upcoming 0 to 2 hours, which significantly impacts both human activities and daily life. However, prevailing deep learning models have primarily focused on a single radar echo data source, limiting their ability to effectively capture intricate and rapidly evolving precipitation patterns. Thus, meteorological satellite is considered to supplement radar echo data. To achieve a comprehensive integration of multi-source data with enhanced details, a two-stage fusion satellite and radar GAN-based prediction network (named FsrGAN) is proposed. In the first stage, we design a satellite-radar fusion prediction network known as FsrNet. This network employs an encoder-fusion-decoder architecture, where a novel spatial-channel attention (SCA) is proposed to enhance the filtering and fusion of multi-source and multi-scale features. In the second stage, we introduce a GAN-based network (FusionGAN) that also mines the complementary information of satellite images to sharpen the first-stage predicted radar maps with more details. Experiments are conducted on meteorological dataset in the Yangtze River Delta (YRD) region. The test results exhibit the notably superior performance of our model in terms of image quality and precipitation forecasting metrics in comparison to traditional optical flow-based methods and some well-known deep learning methods (ConvLSTM, ConvGRU, TrajGRU and PredRNN++). More importantly, our fusion model using satellite and radar data demonstrates the ability to predict convective initiation.
30 August 2023 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Precipitation nowcasting underlying various public services from rainstorm warning to flight safety, is quite important and remains challenging due to the fast change in convective weather. Although some deep learning models have been proposed to make prediction automatically, most of them just deal with a single radar echo data source, making them hard to adapt to heterogeneous and diverse data in practice. In this work, a heterogeneous spatiotemporal attention fusion prediction network (HST-AFP) is proposed for radar echo extrapolatio(deterministic output) and further precipitation nowcasting, which deals with mining and fusing knowledge from multiple heterogeneous spatiotemporal (ST) data sources, including history radar echo observations and numerical weather prediction (NWP) data. With the help of the proposed attention-based ST diffusion module (ASTD), the multi-encoder is designed to extract information from both dense ST tensor and sparse ST tensor. On the other hand, the fusion-decoder achieves very deep trainable residual fusion prediction by integrating scale-wise attention fusion module (SWAF) and deep residual spatial and temporal attention mechanism (DRSTA). It can adaptively blend multi-source ST features and rescale the multi-scale temporal-wise and spatial-wise features for better prediction. Experiments in a real-world dataset of South China show that compared with the ingenious RNN-based methods and newly proposed Unet-based methods, our HST-AFP network can handle complex input with heterogeneity in both space and time domains, and performs better on the precipitation nowcasting metrics, as well as requires remarkable shorter forecast time.