11 September 2023 in IEEE Transactions on Parallel and Distributed Systems
Microservices are an architectural and organizational paradigm for Internet application development. In cloud data centers, delay-sensitive applications receive massive user requests, which are fed into multiple queues and subsequently served by multiple microservice instances. Accordingly, effective deployment of multiple queues and containers significantly reduces queuing delay, processing delay, and communication delay, boosting overall service performance. Due to the increased complexity of call dependencies, probabilistic routing paths, and microservice multiplexing, the deployment of service instances fully interacts with request routing, bringing great difficulties to service orchestration. In this case, it is valuable to simultaneously consider service deployment and request routing in a fine-grained manner. However, most existing studies considered them as two independent components with local optimization, while data dependencies and the deployment of multiple queues/containers are not well handled. Therefore, this paper proposes to jointly optimize the deployment and request routing of microservice call graphs based on fine-grained queuing network analysis and container orchestration. We first formulate the joint optimization problem as a mixed-integer nonlinear program and then exploit open Jackson queuing networks to model the intrinsic data dependencies and analyze response latency and microservice multiplexing. To optimize the overall cost and latency, this paper presents an efficient two-stage heuristic algorithm. The algorithm consists of a resource-splitting-based deployment approach and a partition-mapping-based routing method, which employ instance-level optimization to enhance performance. Further, this paper also provides mathematical analysis on the performance and complexity of the propose algorithm. Finally, comprehensive trace-driven experiments demonstrate that the overall performance of our approach is better than existing microservice benchmarks. Extensive simulation results show that average deployment cost is reduced by 27.4% and the end-to-end response latency is reduced by 15.1% on average.
19 August 2024 in IEEE Transactions on Network and Service Management
Edge computing technologies with container-based microservice architectures promise to provide stable and low-latency services for large-scale and complex edge applications. However, due to the limited CPU and storage resources in edge computing scenarios, the coarse-grained service deployment on edge nodes causes performance bottlenecks. In addition, the effective deployment of microservices is tightly correlated with request routing, but the current research ignores the joint optimization of multi-instance deployment and routing. In this paper, we first model the problem of jointly optimizing service deployment and routing in a dynamically changing environment with multi-edge network collaboration based on a queuing network analysis. Secondly, we design heuristic algorithms to scale microservice instances horizontally in dynamic user request states. In addition, we propose a reinforcement learning algorithm based on reward shaping (RSPPO) to minimize user waiting delay and edge network resource consumption. We also solve the microservice deployment and request routing problem for multi-edge collaboration to achieve load balancing among edge nodes. Finally, extensive experiments verify the significant and extensive effectiveness of our algorithm.
6 March 2025 in IEEE Transactions on Network and Service Management
The service modules of the traditional Mobile Edge Computing (MEC) are difficult to deploy, extend, and maintain in real networks because of the highly sophisticated systems. To promote the generalization, openness, and flexibility of the network edge environment, an increasing number of studies are exploring the integration of microservices with MEC. However, the existing work usually treats microservice deployment and request routing as two separate issues, ignoring the interaction between them. Therefore, this paper focuses on the joint optimization of microservice deployment and request routing in the multi-edge cloud scenarios. We establish a problem model for minimizing the average response latency, considering the transmission of requests across edge clouds. Then, in view of the complexity of the scene, this paper proposes a joint training strategy of microservice deployment and request routing based on deep reinforcement learning and Best Fit Decreasing algorithm. The algorithm takes the change of microservice deployment scheme as the action of the agent, introduces the Best Fit Decreasing algorithm to construct request routing based on the deployment scheme, and calculates rewards using the complete joint microservice deployment and request routing scheme for subsequent network training. Finally, experimental results show that the proposed algorithm can effectively reduce the response time delay and system running power compared with other algorithms.
9 November 2021 in Environmental Science and Pollution Research
10 April 2026 in IEEE Transactions on Network Science and Engineering
Emerging microservice architectures have enabled the efficient development and maintenance of a wide variety of online applications. However, the computationally intensive nature of these applications poses substantial challenges for microservice orchestration in cloud-edge collaborative networks. The frequent communications and complex service dependencies between microservices create tight coupling between deployment and routing, complicating the orchestration process. Furthermore, large-scale stateful microservices often require numerous data queries to cloud-based databases, resulting in significant communication overhead, which adversely affects quality of service (QoS). While many existing studies have focused on isolated aspects of this problem, such as reinforcement learning based orchestration and edge-only scheduling frameworks, a comprehensive solution addressing both computational and database access challenges is still lacking. To bridge this gap, we investigate microservice orchestration in hybrid edge-cloud networks with a focus on database interactions for delay-sensitive applications. We propose a tree-based application model and multi-instance queuing networks for precise delay analysis. To optimize performance, we introduce the Tree Structure Chain Serialization (TCSVN) algorithm, integrated with reinforcement learning using truncated Gaussian Thompson sampling and a step-updating mechanism. Finally, experimental results demonstrate that our approach improves end-to-end latency by 32%-45% and reduces deployment cost by 14%-26% compared to existing baseline methods across multiple performance metrics.