The recent wave of digitalization is characterized by the widespread deployment of sensors in many different environments, e.g., multi-sensor systems represent a critical enabling technology towards full autonomy in industrial scenarios. Sensors usually produce vast amounts of unlabeled data in the form of multivariate time series that may capture normal conditions or anomalies. Multivariate Time Series Anomaly Detection (MTSAD), i.e., the ability to identify normal or irregular operative conditions of a system through the analysis of data from multiple sensors, is crucial in many fields. However, MTSAD is challenging due to the need for simultaneous analysis of temporal (intra-sensor) patterns and spatial (inter-sensor) dependencies. Unfortunately, labeling massive amounts of data is practically impossible in many real-world situations of interest (e.g., the reference ground truth may not be available or the amount of data may exceed labeling capabilities); therefore, robust unsupervised MTSAD is desirable. Recently, advanced techniques in machine learning and signal processing, including deep learning methods, have been developed for unsupervised MTSAD. In this article, we provide an extensive review of the current state of the art with a theoretical background about multivariate time-series anomaly detection. A detailed numerical evaluation of 13 promising algorithms on two publicly available multivariate time-series datasets is presented, with advantages and shortcomings highlighted.
Multi-sensor anomaly detection plays a crucial role in several applications, including industrial monitoring, network intrusion detection, and healthcare monitoring. However, the task poses significant challenges due to the presence of massive unlabeled data, the difficulty of identifying normal patterns in the spatio-temporal data, and the inherent complexity of defining an anomaly. Moreover, noisy sensor measurements could potentially result in models erroneously detecting noise as an anomaly, and the existence of different types of anomalies adds to the complexity. Existing multi-sensor anomaly detection methods are mostly designed for labeled datasets and often disregard crucial factors such as spatio-temporal dependencies, noise presence in training data, and the existence of multiple types of anomalies; thus, their applicability is limited. In this paper, we propose a novel framework called multi-objective transformer networks for anomaly detection (MTAD) that leverages the power of transformer architectures and optimal truncated singular value decomposition (OT-SVD) for robust unsupervised multi-sensor anomaly detection. MTAD comprises a multi-head transformer encoder for effective time series representation learning, a convolutional decoder for reconstruction, and a memory network for predictive analysis. The model processes denoised (via OT-SVD) input through the network and computes both reconstruction and prediction losses. MTAD jointly optimizes the modules in an end-to-end mechanism to minimize the combined weighted loss. We compare MTAD with other state-of-the-art methods using several metrics and demonstrate that our approach outperforms existing solutions. Furthermore, we conducted an ablation to demonstrate the contribution of each module to the overall performance.
In the era of data-driven decision-making, multi-sensor systems acquire complex, high-dimensional streams capturing temporal dynamics, and multivariate time series anomaly detection has become significantly relevant in several application domains. Conventional methods relying on supervised and semi-supervised learning require labeled data, which might not be available in various scenarios. Conversely, noise and outliers present in real-world sensor measurements negatively impact unsupervised methods. Furthermore, several methods rely on black-box architectures, which limit their use in safety-critical applications where interpretability and explainability are often necessary. To address these challenges, we propose a novel unsupervised multivariate time series anomaly detection method that exploits low-rank and sparse (LRS) decomposition combined with spectral detection. More specifically, we employ augmented Lagrange multiplier-based optimization with eigenvalue soft thresholding for decomposition. Data points are projected onto a low-dimensional subspace, capturing the underlying data structure and enabling robust anomaly detection in noisy multi-sensor environments. Finally, the effectiveness of the proposed approach is presented via performance comparison with several existing methods using publicly-available datasets collecting real-world sensor measurements from testbeds of water treatment systems.
18 December 2024 in IEEE Signal Processing Letters
Anomaly detection in multivariate time series (MTS) is crucial in domains such as industrial monitoring, cybersecurity, healthcare, and autonomous driving. Deep learning approaches have improved anomaly detection but lack interpretability. We propose an explainable anomaly detection (XAD) framework using a sparse non-linear vector autoregressive network (SNL-VAR-Net). This framework combines neural networks with vector autoregression for non-linear representation learning and interpretable models. We employ regularization to enforce sparsity, enabling efficient handling of long-range dependencies. Additionally, augmented Lagrange multiplier-based techniques for low-rank and sparse decomposition reduce the impact of noise. Evaluation on publicly available datasets shows that SNL-VAR-Net offers comparable performance to deep learning methods with better interpretability.
Anomaly detection in multivariate time series (MTS) from sensor data is critical in many industrial applications. The challenge lies in managing massive unlabeled datasets with complex spatio-temporal correlations, diverse anomalies, and noise. While several unsupervised methods have been proposed, they are often limited to specific applications. In this paper, we introduce a probabilistic self-supervised framework, Autoregressive Density Estimation Transformer (ADET). ADET integrates an efficient transformer for learning spatio-temporal representations with density estimation networks for multi-score anomaly detection, focusing on point-to-point, point-to-distribution, and distribution-to-distribution distances. ADET improves noise resilience using optimal truncated singular value decomposition (OT-SVD) in an end-to-end optimization process. We conducted experiments by employing several encoders and performed an ablation study to examine the effect of OT-SVD.
In this paper, we propose a novel modular architecture for self-supervised multi-sensor anomaly detection and localization. The framework consists of a spatio-temporal encoder for representation learning, a decoder for latent reconstruction, a predictive memory network for sub-sequence pattern identification, and a denoiser for false-positive reduction. It uniquely combines a reconstruction and latent prediction network and optimizes the modules in an end-to-end mechanism to minimize the combined weighted loss. We demonstrate the flexibility and efficiency of our architecture by introducing different components for each module, showcasing its adaptability and enhanced performance in anomaly detection and localization.
About Me
Department of Information Theory, Simula UiB, Norway
Department of Information Theory, Simula UiB, Norway