6 September 2018 in Space Weather
Accurate forecasting of the properties of coronal mass ejections (CMEs) as they approach Earth is now recognized as an important strategic objective for both NOAA and NASA. The time of arrival of such events is a key parameter, one that had been anticipated to be relatively straightforward to constrain. In this study, we analyze forecasts submitted to the Community Coordinated Modeling Center at NASA's Goddard Space Flight Center over the last 6 years to answer the following questions: (1) How well do these models forecast the arrival time of CME‐driven shocks? (2) What are the uncertainties associated with these forecasts? (3) Which model(s) perform best? (4) Have the models become more accurate during the past 6 years? We analyze all forecasts made by 32 models from 2013 through mid‐2018, and additionally focus on 28 events, all of which were forecasted by six models. We find that the models are generally able to predict CME‐shock arrival times—in an average sense—to within ±10 hr, but with standard deviations often exceeding 20 hr. The best performers, on the other hand, maintained a mean error (bias) of −1 hr, a mean absolute error of 13 hr, and a precision (standard deviation) of 15 hr. Finally, there is no evidence that the forecasts have become more accurate during this interval. We discuss the intrinsic simplifications of the various models analyzed, the limitations of this investigation, and suggest possible paths to improve these forecasts in the future.
1 October 2004 in Solar Physics
31 December 2018 in Space Weather
Geomagnetic indices are convenient quantities that distill the complicated physics of some region or aspect of near‐Earth space into a single parameter. Most of the best‐known indices are calculated from ground‐based magnetometer data sets, such as Dst, SYM‐H, Kp, AE, AL, and PC. Many models have been created that predict the values of these indices, often using solar wind measurements upstream from Earth as the input variables to the calculation. This document reviews the current state of models that predict geomagnetic indices and the methods used to assess their ability to reproduce the target index time series. These existing methods are synthesized into a baseline collection of metrics for benchmarking a new or updated geomagnetic index prediction model. These methods fall into two categories: (1) fit performance metrics such as root‐mean‐square error and mean absolute error that are applied to a time series comparison of model output and observations and (2) event detection performance metrics such as Heidke Skill Score and probability of detection that are derived from a contingency table that compares model and observation values exceeding (or not) a threshold value. A few examples of codes being used with this set of metrics are presented, and other aspects of metrics assessment best practices, limitations, and uncertainties are discussed, including several caveats to consider when using geomagnetic indices.
15 October 2019 in Space Weather
The Community Coordinated Modeling Center has been leading community‐wide space science and space weather model validation projects for many years. These efforts have been broadened and extended via the newly launched International Forum for Space Weather Modeling Capabilities Assessment (https://ccmc.gsfc.nasa.gov/assessment/). Its objective is to track space weather models' progress and performance over time, a capability that is critically needed in space weather operations and different user communities in general. The Space Radiation and Plasma Effects Working Team of the aforementioned International Forum works on one of the many focused evaluation topics and deals with five different subtopics (https://ccmc.gsfc.nasa.gov/assessment/topics/radiation‐all.php) and varieties of particle populations: Surface Charging from tens of eV to 50‐keV electrons and internal charging due to energetic electrons from hundreds keV to several MeVs. Single‐event effects from solar energetic particles and galactic cosmic rays (several MeV to TeV), total dose due to accumulation of doses from electrons (>100 keV) and protons (>1 MeV) in a broad energy range, and radiation effects from solar energetic particles and galactic cosmic rays at aviation altitudes. A unique aspect of the Space Radiation and Plasma Effects focus area is that it bridges the space environments, engineering, and user communities. The intent of the paper is to provide an overview of the current status and to suggest a guide for how to best validate space environment models for operational/engineering use, which includes selection of essential space environment and effect quantities and appropriate metrics.
14 December 2021 in Space Weather
We present a machine‐learning‐based model of relativistic electron fluxes >1.8 MeV using a neural network approach in the Earth's outer radiation belt. The Outer RadIation belt Electron Neural net model for Relativistic electrons (ORIENT‐R) uses only solar wind conditions and geomagnetic indices as input. For the first time, we show that the state of the outer radiation belt can be determined using only solar wind conditions and geomagnetic indices, without any initial and boundary conditions. The most important features for determining outer radiation belt dynamics are found to be AL, solar wind flow speed and density, and SYM‐H indices. ORIENT‐R reproduces out‐of‐sample relativistic electron fluxes with a correlation coefficient of 0.95 and an uncertainty factor of ∼2. ORIENT‐R reproduces radiation belt dynamics during an out‐of‐sample geomagnetic storm with good agreement to the observations. In addition, ORIENT‐R was run for a completely out‐of‐sample period between March 2018 and October 2019 when the AL index ended and was replaced with the predicted AL index (lasp.colorado.edu/home/personnel/xinlin.li). It reproduces electron fluxes with a correlation coefficient of 0.92 and an out‐of‐sample uncertainty factor of ∼3. Furthermore, ORIENT‐R captured the trend in the electron fluxes from low‐earth‐orbit (LEO) SAMPEX, which is a completely out‐of‐sample data set both temporally and spatially. In sum, the ORIENT‐R model can reproduce transport, acceleration, decay, and dropouts of the outer radiation belt anywhere from short timescales (i.e., geomagnetic storms) and very long timescales (i.e., solar cycle) variations.
1 January 2006 in Advances In Space Research