Log anomaly detection is an efficient method to manage modern large-scale Internet of Things (IoT) systems. More and more works start to apply natural language processing (NLP) methods, and in particular word2vec, in the log feature extraction. Word2vec can extract the relevance between words and vectorize the words. However, the computing cost of training word2vec is high. Anomalies in logs are dependent on not only an individual log message but also on the log message sequence. Therefore, the vector of words from word2vec can not be used directly, which needs to be transformed into the vector of log events and further transformed into the vector of log sequences. To reduce computational cost and avoid multiple transformations, in this paper, we propose an offline feature extraction model, named LogEvent2vec, which takes the log event as input of word2vec to extract the relevance between log events and vectorize log events directly. LogEvent2vec can work with any coordinate transformation methods and anomaly detection models. After getting the log event vector, we transform log event vector to log sequence vector by bary or tf-idf and three kinds of supervised models (Random Forests, Naive Bayes, and Neural Networks) are trained to detect the anomalies. We have conducted extensive experiments on a real public log dataset from BlueGene/L (BGL). The experimental results demonstrate that LogEvent2vec can significantly reduce computational time by 30 times and improve accuracy, comparing with word2vec. LogEvent2vec with bary and Random Forest can achieve the best F1-score and LogEvent2vec with tf-idf and Naive Bayes needs the least computational time.
Simultaneous wireless information and power transfer (SWIPT) transmits information and powers wireless nodes with the same radio frequency signal. It can prolong the life time of the energy-constrained wireless nodes. Current works of SWIPT focus on one-hop and two-hop wireless network. In order to verify the performance of SWIPT in multi-hop energy-constrained wireless network (MECWN) where the energy harvested by the receiver node can be as an energy compensation for data forwarding, this paper concurrently considers SWIPT and routing selection in MECWN. To reduce the energy consumption, we first formulate the information and energy allocation problem of link in a forwarding path, which is dependent on the next-hop node, and solve it by an iterative allocation algorithm. A novel routing metric evaluates the energy consumption of link transmitted with or without SWIPT. The energy-aware SWIPT routing algorithm allocates the information and energy of link with allocation algorithm during path finding process. To the best of our knowledge, this is the first solution that takes account of SWIPT and routing in MECWN. Our performance studies demonstrate that our proposed algorithms can effectively exploit those node resources whose energy are not enough and significantly decrease the energy consumption.
1 January 2022 in Computer Systems Science and Engineering
System logs record detailed information about system operation and are important for analyzing the system's operational status and performance. Rapid and accurate detection of system anomalies is of great significance to ensure system stability. However, large-scale distributed systems are becoming more and more complex, and the number of system logs gradually increases, which brings challenges to analyze system logs. Some recent studies show that logs can be unstable due to the evolution of log statements and noise introduced by log collection and parsing. Moreover, deep learning-based detection methods take a long time to train models. Therefore, to reduce the computational cost and avoid log instability we propose a new Word2Vec-based log unsupervised anomaly detection method (LogUAD). LogUAD does not require a log parsing step and takes original log messages as input to avoid the noise. LogUAD uses Word2Vec to generate word vectors and generates weighted log sequence feature vectors with TF-IDF to handle the evolution of log statements. At last, a computationally efficient unsupervised clustering is exploited to detect the anomaly. We conducted extensive experiments on the public dataset from Blue Gene/L (BGL). Experimental results show that the F1-score of LogUAD can be improved by 67.25% compared to LogCluster.
A node can provide a file to other nodes after downloading the file or data from the Internet. When more than one node have obtained the same file, this is considered a multisource transmission, in which all these nodes can act as candidate providers (sources) and transmit the file to a new request node (destination) together. In cases where there is negligible or no interference, multisource transmission can improve the download throughput because of parallel transmissions through multiple paths. However, this improvement is not guaranteed due to wireless interference among different paths. Wireless interference can be alleviated by the multiradio and multichannel technique. Because the source and multipath routing selections interact with channel assignment, the multisource transmission problem with multiradio and multichannel presents a significant challenge. In this paper, we propose a distributed joint source, routing, and channel selection scheme. The source selection issue can be concurrently solved via multipath finding. There are three sub-algorithms in our scheme, namely, interference-aware routing algorithm, channel assignment algorithm, and local routing adjustment algorithm. The interference-aware routing algorithm is used to find paths sequentially and is jointly executed with the channel assignment algorithm. After finding a new path, the local routing adjustment algorithm may be executed to locally adjust the selected paths so as to further reduce wireless interference. Extensive simulations have been conducted to demonstrate that our algorithms can effectively improve the network aggregate throughput, as well as reduce delay and packet loss probability.
6 November 2020 in IEEE Transactions on Industrial Informatics
Intelligent anomaly detection for Key Performance Indicators (KPIs) is important for keeping services reliable in industrial-based Cyber-Physical Systems (CPS). However, it is common in practice for various KPI sampling strategies to be utilized. We experimentally verify that anomaly detection is highly sensitive to irregular sampling, and accordingly go on to investigate low-cost anomaly detection for large-scale irregular KPIs. Irregular KPIs can be classified into four types. We propose an anomaly detection framework based on these irregular types. Moreover, to handle the various lengths and phase shifts, we propose a Normalized version of Unequal Cross-Correlation (NUCC). To avoid high computational costs, we analyze the low-rank feature of KPIs data and propose a matrix factorization based alignment algorithm. Extensive simulations using three public datasets and two real-world datasets demonstrate that our algorithm can achieve a larger F1-score than Minkowski distance and less time than dynamic time warping distance.
12 September 2024 in IEEE Transactions on Information Forensics and Security
Multivariate time series anomaly detection (MTSAD) plays a crucial role in the Internet of Things (IoT) to identify device malfunction or system attacks. Graph neural networks (GNN) are widely applied in MTSAD to capture the spatial features among sensors. However, GNNs depend on a graph structure and explicit graph structures are not always available. To solve the problem of missing explicit graph structure, graph structure learning is introduced to learn an accurate graph structure joint with a GNNs-based anomaly detection task. However, the existing GSL-based methods provide only a partial view of the graph structure and can not represent multiple and complex relationships. The noise of data also brings noisy edges. Therefore, we propose a fusion graph structure learning-based multivariate time-series anomaly detection with structured prior knowledge (FuGLAD). It appears to be the first application of fusion graphs in time series anomaly detection, to the best of our knowledge. FuGLAD selects three kinds of typical graph structure learners to learn as many relationship types among sensors as possible and exploits the prior similarity to evaluate the importance of all learned graphs and adaptively learn the fusion weight instead of the direct average weight. To handle noise in raw data, FuGLAD compares the neighbors of nodes by Jaccard similarity to identify and remove the noisy edges in the prior graph. Extensive experiments demonstrate that our approach outperforms state-of-the-art single-graph structure learning techniques in detection performance across four public and real-world datasets.