Microservices architecture is quickly replacing monolithic and multi-tier architectures as the implementation choice for large-scale web applications as it allows independent development, scalability, and maintenance. However, even with careful node scheduling and scaling, the microservices applications are still vulnerable to performance degradation due to unexpected (dependent or independent) events like anomalous node behavior, workload interference, or sudden spikes in requests or retries. These events can adversely affect the performance of one or more microservices (bottlenecks), degrading the overall application performance. To ensure a good customer experience and avoid revenue loss, it is crucial to detect and mitigate all bottlenecks swiftly. This work introduces GAMMA, a novel, explainable graph learning model that integrates a mixture of experts to detect multiple bottlenecks. We evaluated GAMMA using a popular open-source benchmarking application deployed on Kubernetes under various practical bottleneck scenarios. Our experimental evaluation results show that GAMMA provides significantly better performance (46% higher F1 score) than existing works that employ deep learning, machine learning, and statistical techniques, demonstrating its ability to detect multiple bottlenecks by learning complex interactions in a microservices architecture. The dataset is made publicly available [49] for reproducibility and further research in the field.
Roblox Metaverse supports an impressive 55M daily users. The underlying infrastructure is geographically distributed, with multiple edge data centers in all continents. The Meta-verse stack consists of multiple layers of software systems, with complex dependencies that can be represented as a DAG (Directed Acyclic Graph) or a multi-layered queueing system. In short, the concept of Metastability happens when each moving part of the stack works in harmony. In contrast, a Metastable failure results from a trigger originating in some part of the stack that cascades through multiple dependencies to finally affect a third-party system. Such kind of failure has large-scale consequences and can deem the overall stack unusable. The present talk will deep-dive in the characterization (or absence) of metastability based on a queueing theory model and analyze the dynamics that can lead to different flavors of system-wide failures. Furthermore, the model will be used as the basis to (mathematically) devise measures for the early detection of failures and explore fault-tolerance measures that ensure metastability.
When one hears the word Metaverse, it is automatically associated with millions of users, immersive experiences and its potential to change our lives. But, what enables the Metaverse to function at such a scale? This talk will present the different challenges associated with handling 55 million daily users within the Roblox Metaverse. From addressing user Quality of Experience, to distributed architecture, programming model and scheduling, we will cover the entire stack underneath. In particular, I will put special attention into the AI-Metaverse relationship. On one hand, a large proportion of workloads are based on one or multiple ML/DL models and I will present the challenges of scaling them. On the other hand, I will explore infrastructure and service model challenges that can be addressed with AI, e.g. multi-resource and multi-datacenter scheduling with Reinforcement learning.