The numerous recent breakthroughs in machine learning make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in Space Weather. The purpose is twofold. On one hand, we will discuss previous works that use machine learning for Space Weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the Space Weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics‐based and machine learning approaches, known as gray box.

In this study, we present a method that combines a Long Short‐Term Memory (LSTM) recurrent neural network with a Gaussian process (GP) model to provide up to 6‐hr‐ahead probabilistic forecasts of the Dst geomagnetic index. The proposed approach brings together the sequence modeling capabilities of a recurrent neural network with the error bars and confidence bounds provided by a GP. Our model is trained using the hourly OMNI and Global Positioning System (GPS) databases, both of which are publicly available. We first develop a LSTM network to get a single‐point prediction of Dst. This model yields great accuracy in forecasting the Dst index from 1 to 6 hr ahead, with a correlation coefficient always higher than 0.873 and a root‐mean‐square error lower than 9.86. However, even if global metrics show excellent performance, it remains poor in predicting intense storms (Dst < −250 nT) 6 hr in advance. To improve it and to obtain probabilistic forecasts, we combine the LSTM model obtained with a GP and evaluate the hybrid predictor using the receiver operating characteristic curve and the reliability diagram. We conclude that this hybrid methodology provides improvements in the forecast of geomagnetic storms, from 1 to 6 hr ahead.

The solar wind‐magnetosphere system is nonlinear. The solar wind drivers of geosynchronous electrons with energy range of 1.8–3.5 MeV are investigated using mutual information, conditional mutual information (CMI), and transfer entropy (TE). These information theoretical tools can establish linear and nonlinear relationships as well as information transfer. The information transfer from solar wind velocity (V_sw) to geosynchronous MeV electron flux (J_e) peaks with a lag time of 2 days. As previously reported, J_e is anticorrelated with solar wind density (n_sw) with a lag of 1 day. However, this lag time and anticorrelation can be attributed at least partly to the J_e(t + 2 days) correlation with V_sw(t) and n_sw(t + 1 day) anticorrelation with V_sw(t). Analyses of solar wind driving of the magnetosphere need to consider the large lag times, up to 3 days, in the (V_sw, n_sw) anticorrelation. Using CMI to remove the effects of V_sw, the response of J_e to n_sw is 30% smaller and has a lag time < 24 h, suggesting that the MeV electron loss mechanism due to n_sw or solar wind dynamic pressure has to start operating in < 24 h. n_sw transfers about 36% as much information as V_sw (the primary driver) to J_e. Nonstationarity in the system dynamics is investigated using windowed TE. When the data are ordered according to transfer entropy value, it is possible to understand details of the triangle distribution that has been identified between J_e(t + 2 days) versus V_sw(t).

We report the properties of a novel type of sub-proton scale magnetic hole found in two dimensional particle-in-cell simulations of decaying turbulence with a guide field. The simulations were performed with a realistic value for ion to electron mass ratio. These structures, electron vortex magnetic holes (EVMHs), have circular cross-section. The magnetic field depression is associated with a diamagnetic azimuthal current provided by a population of trapped electrons in petal-like orbits. The trapped electron population provides a mean azimuthal velocity and since trapping preferentially selects high pitch angles, a perpendicular temperature anisotropy. The structures arise out of initial perturbations in the course of the turbulent evolution of the plasma, and are stable over at least 100 electron gyroperiods. We have verified the model for the EVMH by carrying out test particle and PIC simulations of isolated structures in a uniform plasma. It is found that (quasi-)stable structures can be formed provided that there is some initial perpendicular temperature anisotropy at the structure location. The properties of these structures (scale size, trapped population, etc.) are able to explain the observed properties of magnetic holes in the terrestrial plasma sheet. EVMHs may also contribute to turbulence properties, such as intermittency, at short scale lengths in other astrophysical plasmas.