The continuous growth of geophysical observations requires adequate methods for their processing and analysis. This becomes one of the most important and widely discussed issues in the data science community. The system analysis methods and data mining techniques are able to sustain the solution of this problem. This paper presents an innovative holistic hardware/software system (HSS) developed for efficient management and intellectual analysis of geomagnetic data, registered by Russian geomagnetic observatories and international satellites. Geomagnetic observatories that comprise the International Real-time Magnetic Observatory Network (INTERMAGNET) produce preliminary (raw) and definitive (corrected) geomagnetic data of the highest quality. The designed system automates and accelerates routine production of definitive data from the preliminary magnetograms, obtained by Russian observatories, due to implemented algorithms that involve artificial intelligence elements. The HSS is the first system that provides sophisticated automatic detection and multi-criteria classification of extreme geomagnetic conditions, which may be hazardous for technological infrastructure and economic activity in Russia. It enables the online access to digital geomagnetic data, its processing results and modelling calculations along with their visualization on conventional and spherical screens. The concept of the presented system agrees with the accepted ‘four Vs’ paradigm of Big Data. The HSS can increase significantly the ‘velocity’ and ‘veracity’ features of the INTERMAGNET system. It also provides fusion of large sets of ground-based and satellite geomagnetic data, thus facilitating the ‘volume’ and ‘variety’ of handled data

The K index is one of the oldest universal indices of geomagnetic activity, introduced in 1938 by Julius Bartels, that is still being widely used. Up to the present day, long-term time series of homogeneous K index records have been accumulated at data repositories all over the world. The multidecadal practice of its application makes it an indispensable source of information for the retrospective analysis of solar–terrestrial interaction for nearly eight solar cycles. Most significantly, while studying the historical geomagnetic data, K index data sheets are in certain cases far easier for automated analysis than the conventional analogue magnetograms. The presented collection includes the results of the K index determination at 41 geomagnetic observatories of the former USSR for the period from July 1957 to the early 1990s. This unique collection was formed at the World Data Center for Solar-Terrestrial Physics in Moscow. The historical data, which are offered to the international scientific community, cover the second half of the 20th century and can be used for the retrospective analysis and study of geomagnetic events in the past, as well as for data validation or forecasting (Sergeyeva et al., 2020). The dataset is available at: https://doi.org/10.1594/PANGAEA.922233, last access: 16 September 2020.

Since June 2012 the Saint Petersburg magnetic observatory is being developed and maintained by two institutions of the Russian Academy of Sciences (RAS) – the Geophysical Center of RAS (GC RAS) and the Saint Petersburg branch of the Pushkov Institute of Terrestrial Magnetism, Ionosphere and Radio Wave Propagation of RAS (IZMIRAN SPb). On 29 April 2016 the application of the Saint Petersburg observatory (IAGA code SPG) for introduction into the INTERMAGNET network was accepted after approval by the experts of the first definitive dataset over 2015, produced by the GC RAS, and on 9 June 2016 the SPG observatory was officially certified. One of the oldest series of magnetic observations, originating in 1834, was resumed in the 21st century, meeting the highest quality standards and all modern technical requirements. In this paper a brief historical and scientific background of the SPG observatory foundation and development is given, the stages of its renovation and upgrade in the 21st century are described, and information on its current state is provided. The first results of the observatory functioning are discussed and geomagnetic variations registered at the SPG observatory are assessed and compared with geomagnetic data from the INTERMAGNET observatories located in the same region.