Detecting anomalies in streaming data is an important issue for many application domains, such as cybersecurity, natural disasters, or bank frauds. Different approaches have been designed in order to detect anomalies: statistics-based, isolation-based, clustering-based, etc. In this paper, we present a structured survey of the existing anomaly detection methods for data streams with a deep view on Isolation Forest (iForest). We first provide an implementation of Isolation Forest Anomalies detection in Stream Data (IForestASD), a variant of iForest for data streams. This implementation is built on top of scikit-multiflow (River), which is an open source machine learning framework for data streams containing a single anomaly detection algorithm in data streams, called Streaming half-space trees. We performed experiments on different real and well known data sets in order to compare the performance of our implementation of IForestASD and half-space trees. Moreover, we extended the IForestASD algorithm to handle drifting data by proposing three algorithms that involve two main well known drift detection methods: ADWIN and KSWIN. ADWIN is an adaptive sliding window algorithm for detecting change in a data stream. KSWIN is a more recent method and it refers to the Kolmogorov–Smirnov Windowing method for concept drift detection. More precisely, we extended KSWIN to be able to deal with n-dimensional data streams. We validated and compared all of the proposed methods on both real and synthetic data sets. In particular, we evaluated the F1-score, the execution time, and the memory consumption. The experiments show that our extensions have lower resource consumption than the original version of IForestASD with a similar or better detection efficiency.
With the advent of Big Data, we are witnessing a rapid and varied production of huge amounts of sequential data that can have multiple dimensions. We are talking about data streams. The characteristics of these data streams make their processing and storage very difficult and at the same time their querying a posteriori. Thus, to deal with this multidimensional aspect of data streams, new systems have been carried out. These propositions have a lot of advantages. However, they do not always enable to best meet the constraints subjected to data streams, such as processing and storage costs. In this sense, we propose in this paper a new generic model for summarizing multidimensional data streams and to deal with storage and processing constraints. This solution uses cascading cubes with different time window hierarchies combined with customizable aggregation and propagation functions for the processing and the storage of multidimensional data streams. Experimentations have been performed with electrical data stream and result were satisfactory.
Smart meters play a central role in the continuous collection of large measurements generated by power grid equipment. These measures are essential to facilitate decision-making. However, the challenge lies in the storage and processing resources making existing solutions less compatible with the dynamic nature of data stream. This paper aims to overcome these constraints by extending and evaluating the efficiency of data stream summaries through the use of advanced machine learning algorithms. Specifically, LSTM, ANN and MLP were applied on these summaries constructed from authentic electrical data streams to predict peaks demand. The results show that these summaries produce insights closely aligned with those obtained with the detailed data for all algorithms used especially with ANN. Similarly, data stream summaries can better adapt to the lack of available storage and processing resources in order to be efficient in exploiting this data for decision-making.