20 June 2022 in IEEE Robotics and Automation Letters
As industry rapidly shifts towards mass personalisation, the need for a decentralised multi-agent system capable of dynamic flexible job shop scheduling (FJSP) is evident. Traditional heuristic and meta-heuristic scheduling methods cannot achieve satisfactory results and have limited application to static environments. Recent Reinforcement Learning (RL) approaches that consider dynamic FJSP, lack flexibility and autonomy as they use a single-agent centralised model, assuming global observability. As such, we propose a Multi-Agent Reinforcement Learning (MARL) system for scheduling dynamically arriving assembly jobs in a robot assembly cell. We applied a Double DQN-based algorithm and proposed a generalised observation, action and reward design for the dynamic FJSP setting. Using a centralised training phase, eEach agent (i.e., robot) in the assembly cell executesmakes decentralised scheduling decisions based on local observations. Our solution demonstrated improved performance against rule-based heuristic methods, for optimising makespan. We also reported the impact of different observation sizes of each agent on optimisation performance.
17 March 2020 in IEEE Transactions on Parallel and Distributed Systems
Enterprise application providers are increasingly moving their workloads to the cloud for technical and economic benefits. Multi-cloud environment makes it possible to orchestrate multiple cloud resources. With the increasing number of available cloud resources provided by multiple cloud providers at different locations with different prices, application providers face the challenge to select proper cloud resources to deploy their applications in the form of a workflow of component service units. Existing studies usually consider minimizing execution time or/and deployment cost. From the perspective of application providers, however, they also pay huge attention to application response time, including particularly network latency between deployed services and users. Meanwhile, application deployment is often subject to stringent budgetary control to ensure financial viability. This paper studies a new type of composite application deployment problem that jointly considers both the performance optimization and budget control in multi-cloud at the global scale. To find solutions with minimal response time without running into the risk of over-spending, we propose a hybrid GA-based approach, featuring new design of domain-tailored service clustering, repair algorithm, solution representation, population initialization, and genetic operators. Extensive experiments using the real-world dataset demonstrate that our proposed hybrid GA approach outperforms some recently proposed approaches.
11 February 2020 in IEEE Transactions on Network and Service Management
10 December 2021 in IEEE Transactions on Parallel and Distributed Systems
Multi-cloud is becoming a popular cloud ecosystem because it allows enterprise users to share the workload across multiple cloud service providers to achieve high-quality services with lower operation cost and higher application resilience. In multi-cloud, cloud services are widely distributed at different locations with differentiated prices. Therefore, Web application providers face the challenge to select proper cloud services for application replication and deployment with the aim of minimizing the deployment cost. Meanwhile, the deployed application replicas must satisfy the constraint on request response time to maintain the quality of user experience. To meet the two major requirements, this article studies a new problem of Web application replication and deployment in multi-cloud (WARDMC) that jointly considers both the cost minimization and constraints on average response time, including particularly request processing time and network latency. To address the problem, we develop a new approach named MCApp. MCApp combines iterative mixed integer linear programming with domain-tailored large neighborhood search to optimize both application replicas deployment and user requests dispatching. Extensive experiments using the real-world datasets demonstrate that MCApp significantly outperforms several recently proposed approaches.
14 December 2023 in Computers & Industrial Engineering