This paper proposes a multimodal and multi-task deep-learning model for instantaneous precipitation rate estimation. Using both thermal infrared satellite radiometer and automatic rain gauge measurements as input, our encoder–decoder convolutional neural network performs a multiscale analysis of these two modalities to estimate simultaneously the rainfall probability and the precipitation rate value. Precipitating pixels are detected with a Probability Of Detection (POD) of 0.75 and a False Alarm Ratio (FAR) of 0.3. Instantaneous precipitation rate is estimated with a Root Mean Squared Error (RMSE) of 1.6 mm/h.

To improve precipitation estimation accuracy, new methods, which are able to merge different precipitation measurement modalities, are necessary. In this study, we propose a deep learning method to merge rain gauge measurements with a ground-based radar composite and thermal infrared satellite imagery. The proposed convolutional neural network, composed of an encoder–decoder architecture, performs a multiscale analysis of the three input modalities to estimate simultaneously the rainfall probability and the precipitation rate value with a spatial resolution of 2 km. The training of our model and its performance evaluation are carried out on a dataset spanning 5 years from 2015 to 2019 and covering Belgium, the Netherlands, Germany and the North Sea. Our results for instantaneous precipitation detection, instantaneous precipitation rate estimation, and for daily rainfall accumulation estimation show that the best accuracy is obtained for the model combining all three modalities. The ablation study, done to compare every possible combination of the three modalities, shows that the combination of rain gauges measurements with radar data allows for a considerable increase in the accuracy of the precipitation estimation, and the addition of satellite imagery provides precipitation estimates where rain gauge and radar coverage are lacking. We also show that our multi-modal model significantly improves performance compared to the European radar composite product provided by OPERA and the quasi gauge-adjusted radar product RADOLAN provided by the DWD for precipitation rate estimation.

In the coming years, Artificial Intelligence (AI), for which Deep Learning (DL) is an essential component, is expected to transform society in a way that is compared to the introduction of electricity or the introduction of the internet. The high expectations are founded on the many impressive results of recent DL studies for AI tasks (e.g. computer vision, text translation, image or text generation...). Also for weather and climate observations, a large potential for AI application exists. We present the results of the recent paper [Moraux et al, 2019], which is one of the first demonstrations of the application of cutting edge deep learning techniques to a practical weather observation problem. We developed a multiscale encoder-decoder convolutional neural network using the three most relevant SEVIRI/MSG spectral images at 8.7, 10.8 and 12.0 micron and in situ rain gauge measurements as input. The network is trained to reproduce precipitation measured by rain gauges in Belgium, the Netherlands and Germany. Precipitating pixels are detected with a POD of 0.75 and a FAR of 0.3. Instantaneous precipitation rate is estimated with a RMSE of 1.6 mm/h. Reference:[Moraux et al, 2019] Moraux, A.; Dewitte, S.; Cornelis, B.; Munteanu, A. Deep Learning for Precipitation Estimation from Satellite and Rain Gauges Measurements. Remote Sens. 2019, 11, 2463.

Seamless prediction systems provide frequently updated forecasts across different timescales by combining observations, such as weather radar data, with numerical weather prediction (NWP) models. These systems are increasingly needed by users like hydrological services, local authorities, renewable energy operators, and smartphone apps to make better and earlier decisions. This is especially true for precipitation, which is highly variable in space and time and strongly influences downstream models like (urban) hydrology. To achieve this, forecasts must not only be fast and accurate but also come with calibrated ensembles to estimate uncertainty and propagate errors properly.In Belgium, Project IMA (inspired by the Japanese word for "now" or "soon") is the seamless prediction system developed by the Royal Meteorological Institute (RMI). It uses RMI’s observation network, including RADQPE for gauge-corrected precipitation estimates, the pysteps-be probabilistic rainfall nowcasting system, the INCA-BE nowcasting system, and the ACCORD NWP models ALARO and AROME. Unlike many other systems, Project IMA offers seamless ensemble precipitation nowcasts for lead times up to 6 hours, updated every 5 minutes, designed to improve flash flood predictions and quantify their uncertainty.This presentation will showcase recent developments in Project IMA, including updates to the open-source pysteps framework, such as an improved runtime efficiency, code structure and better representation of extremes. We will discuss new deep learning-based methods for blending forecasts to extend their lead time and improve accuracy, calibration, and usefulness for end users such as hydrologists, crisis managers and water authorities.Project IMA aims to ensure a rapid transfer from research to operations and encourages open-source contributions to ensure transparency and reproducibility. It supports the United Nations’ “Early Warnings for All” initiative, which strives to make forecasts more accessible and actionable by 2027.

In July 2021, western Europe has been subject to extreme rainfall that lead to severe flooding, incurring heavy property losses and claiming dozens of people’s lives in both Germany and Belgium. This unfortunate disaster is a reminder that carrying out studies about extreme event forecasting is a matter of prime importance in the field of meteorology. The extreme rainfall from July 2021 invites us to study the performance of our deep learning method for precipitation estimation in case of extreme events.The main novelty of our method resides in its ability to merge different physical measurement modalities in order to improve precipitation estimation accuracy. In specific, the proposed method merges rain gauge measurements with a ground-based radar composite and thermal infrared satellite imagery. The proposed convolutional neural network design, composed of an encoder–decoder architecture, performs multiscale analysis of the three input modalities to simultaneously estimate the rainfall probability and the precipitation rate with a spatial resolution of 2 km. The training of our model and its performance evaluation are carried out on a dataset spanning 5 years from 2015 to 2019 and covering Belgium, the Netherlands, Germany and the North Sea. Once trained, we evaluate the performance of our model to estimate the extreme precipitation that happened in Belgium and Germany in July 2021 by comparing our results with the measurements from rain gauges and radar estimation.

Deep-Learning (DL) is a sub-field of Machine-Learning (ML) whose popularity has grown exponentially during the last decade thanks to its numerous successes related to artificial intelligence, e.g. visual object recognition and detection, image generation, natural language processing or speech recognition. But, like other ML algorithms, DL can be applied to a broader variety of problems, as long as the necessary data is available to train the model. In this aspect, the field of meteorology satisfy the data requirement. Particularly, DL could improve the accuracy of precipitation estimation. Accurate precipitation estimation is a very important product of weather institutes for water supply monitoring, and as input and validation for Numerical Weather Prediction models and precipitation nowcasting. Currently, precipitation are mainly estimated using both radar and rain-gauge data, but an important limitation of this method is the limited coverage of these measurements. A possible solution to this problem could come from satellite radiometers observations. Unfortunately, estimating precipitation accurately from satellite radiometer data is challenging. As a solution, we developed a DL method to merge rain gauge measurements with a ground-based radar composite and satellite radiometer imagery. The proposed convolutional neural network, composed of an encoder–decoder architecture, performs a multiscale analysis of the three input modalities to estimate simultaneously the rainfall probability and the precipitation rate. We used SEVIRI infrared channels, the OPERA radar composite and the measurements of automatic rain gauges. The training of our model and its performance evaluation are carried out on a dataset spanning 5 years from 2015 to 2019 and covering Belgium, the Netherlands, Germany and the North Sea. Our results for instantaneous precipitation detection, instantaneous precipitation rate estimation, and for daily rainfall accumulation estimation show that the best accuracy is obtained for the model combining all three modalities. We show that the combination of rain gauge measurements with radar data allows for a considerable increase in the accuracy of the precipitation estimation, and the addition of satellite imagery provides precipitation estimates where rain gauge and radar coverage are lacking. We also show that our multi-modal model significantly improves performance compared to the European radar composite product provided by OPERA and the quasi gauge-adjusted radar product RADOLAN provided by the DWD for precipitation rate estimation. Additionally, we carried a study of our method on the case of the extreme precipitation event of July 2021 that affected Belgium and Germany, which caused huge societal and economical damage.