Effective log anomaly detection can help operators locate and solve problems quickly, ensure the rapid recovery of the system, and reduce economic losses. However, recent log anomaly detection studies have shown some drawbacks, such as concept drift, noise problems, and fuzzy feature relation extraction, which cause data instability and abnormal misjudgment, leading to significant performance degradation. This paper proposes a multi-feature deep fusion of an unstable log anomaly detection model (MDFULog) for the above problems. The MDFULog model uses a novel log resolution method to eliminate the dynamic interference caused by noise. This paper proposes a feature enhancement mechanism that fully uses the correlation between semantic information, time information, and sequence features to detect various types of log exceptions. The introduced semantic feature extraction model based on Bert preserves the semantics of log messages and maps them to log vectors, effectively eliminating worker randomness and noise injection caused by log template updates. An Informer anomaly detection classification model is proposed to extract practical information from a global perspective and predict outliers quickly and accurately. Experiments were conducted on HDFS, OpenStack, and unstable datasets, showing that the anomaly detection method in this paper performs significantly better than available algorithms.
Detailed information on system operation is recorded by system logs, from which fast and accurate detection of anomalies is conducive to service management and system maintenance. Log anomaly detection methods often only handle a single type of anomaly, and the utilization of log messages could be higher, which makes it challenging to improve the performance of log anomaly detection models. This article presents the LTAnomaly model to accomplish log anomaly detection using semantic information, sequence relationships, and component values to make a vector representation of logs, and we add Transformer with long short-term memory (LSTM) as our final classification model. When sequences are processed sequentially, the model is also influenced by the information from the global information, thus increasing the dependence on feature information. This improves the utilization of log messages with a flexible, simple, and robust model. To evaluate the effectiveness of our method, experiments are performed on the HDFS and BGL datasets, with the F1-measures reaching 0.985 and 0.975, respectively, showing that the proposed method enjoys higher accuracy and a more comprehensive application range than existing models.