Nowadays, increasing business applications running in the cloud are embracing the microservice architecture. This article presents the challenges and implications of diagnosing root causes of anomalies in large-scale microservice architecture using real incidents in IBM Bluemix. We propose ServiceRank, a novel framework for anomaly detection and root cause identification in the microservice architecture to tackle these challenges. ServiceRank introduces an anomaly detector followed by a root cause analysis module, which detects the suspected abnormal service without pre-defined thresholds. To generalize our approach, we design a causal relationship extraction approach to construct impact graphs for root cause investigation according to specific anomalies. To eliminate cloud design-patterns' impact on anomaly diagnosis, we propose a correlation calibration mechanism in ServiceRank and present a calibration algorithm for the circuit breaker - A typical protection pattern in the microservice architecture. Finally, we design a heuristic investigation algorithm based on the second-order random walk to identify the anomaly's root cause. Experimental results in a simulated environment and the IBM Bluemix platform show that ServiceRank outperforms selected approaches in accuracy and offers fast identification of root cause service when an anomaly occurs. Moreover, we can deploy ServiceRank rapidly and easily in various systems without any pre-defined knowledge.
8 May 2020 in IEEE Transactions on Services Computing
This paper presents a multi-metric and self-adaptive root cause diagnosis framework, named MS-Rank. MS-Rank decomposes the task into four phases: impact graph construction, random walk diagnosis, result precision evaluation, metrics weight update. Initially, we introduce the concept of implicit metrics and propose a composite impact graph construction algorithm, using multiple types of metrics to discover causal relationships between services. Afterwards, we propose a diagnostic algorithm in which forward, selfward and backward transitions are designed to heuristically identify the root cause services. In addition, we establish a self-adaptive mechanism to update the confidence of different metrics dynamically according to their diagnostic precision. Lastly, we develop a prototype system and integrate MS-Rank into real production system - IBM Cloud. Experimental results show that MS-Rank has a high diagnostic precision and its performance outperforms several selected benchmarks. Through multiple rounds of diagnosis, MS-Rank can optimize itself effectively. MS-Rank can be rapidly deployed in various microservice-based systems and applications, requiring no predefined knowledge. MS-Rank also allows us to introduce expert experiences into its framework to improve the diagnostic efficiency and precision.
22 November 2017 in IEEE Transactions on Industrial Informatics
Energy efficiency raises significant concerns as it is one of the most promising ways to mitigate climate change. Disaggregation and identification of individual electrical appliances activities is one of the essentials for energy preservation especially for smart buildings. This paper proposes a lightweight electrical appliance activity detection approach for smart building, which leverages a single smart metering device to establish a learning and detection processing for multiple appliances. In this system, data interpolation and transition detection algorithm are proposed to effectively reduce the cost of model training and optimize the detection accuracy. The concept of appliance fingerprint is proposed and a variety of fingerprints, including appliance-based and context-based, are defined to depict fine-grained appliance characteristics. Based on these fingerprints, the paper proposes a multi-source fingerprint-weighting KNN (FWKNN) classification algorithm and presents a boosting framework for continuous online learning and detection. A prototype system is implemented and demonstrated in IBM Bluemix PaaS cloud platform. Experimental result and analysis prove that FWKNN outperforms other benchmark methods in detection accuracy.