To meet the explosive growth of mobile traffic, the 5G network is designed to be flexible and support multi-access edge computing (MEC), thereby improving the end-to-end quality of service (QoS). In particular, 5G network slicing, which allows a physical infrastructure to split into multiple logical networks, keeps the balance of network resource allocation among different service types with on-demand resource requests. However, achieving effective resource allocation across the end-to-end network is difficult due to the dynamic characteristics of slicing requests such as uncertain real-time resource demand and heterogeneous requirements. In this paper, we develop a reinforcement learning (RL)-based dynamic resource allocation framework for end-to-end network slicing with heterogeneous requirements in multi-layer MEC environments. We first design a hierarchical MEC architecture and formulate a resource allocation problem for the end-to-end network slicing as an optimization problem using the Markov decision process (MDP). Using proximal policy optimization (PPO), we develop independently-collaborative and jointly-collaborative dynamic resource allocation algorithms to maximize resource efficiency while satisfying the QoS of slices. Experimental results show that the proposed algorithms can recognize the characteristics of slice requests and coming resource demands and efficiently allocate resources with a high QoS satisfaction rate.

Network slicing to create multiple virtual networks, called network slice, is a promising technology to enable networking resource sharing among multiple tenants for the 5th generation (5G) networks. By offering a network slice to slice tenants, network slicing supports parallel services to meet the service level agreement (SLA). In legacy networks, every tenant pays a fixed and roughly estimated monthly or annual fee for shared resources according to a contract signed with a provider. However, such a fixed resource allocation mechanism may result in low resource utilization or violation of user quality of service (QoS) due to fluctuations in the network demand. To address this issue, we introduce a resource management system for network slicing and propose a dynamic resource adjustment algorithm based on reinforcement learning approach from each tenant’s point of view. First, the resource management for network slicing is modeled as a Markov Decision Process (MDP) with the state space, action space, and reward function. Then, we propose a Q-learning-based dynamic resource adjustment algorithm that aims at maximizing the profit of tenants while ensuring the QoS requirements of end-users. The numerical simulation results demonstrate that the proposed algorithm can significantly increase the profit of tenants compared to existing fixed resource allocation methods while satisfying the QoS requirements of end-users.

We consider a virtualized edge-computing infrastructure for drone applications, in which a virtualized container running on an edge node controls drones and a software-defined network provides a network connectivity between the drones and their virtualized controllers. We propose a seamless migration scheme that migrates a virtualized drone controller to an edge node that is close to its associated drone without suspending the drone control in the edge-computing infrastructure.

Vehicular communication is revolutionizing transportation by enhancing passenger experience and improving safety through seamless message exchanges with nearby vehicles and roadside units (RSUs). To accommodate the growing number of vehicles in dense urban traffic with limited channel availability, non-orthogonal multiple access (NOMA) is a promising solution due to its ability to improve spectral efficiency by sharing channels among multiple users. However, to completely leverage NOMA on mobile vehicular networks, a chain of operations and resources must be optimized, including vehicle user (VU) and RSU association, channel assignment, and optimal power control. In contrast, traditional orthogonal multiple access (OMA) allocates separate channels to users, simplifying management but falling short in high-density environments. Additionally, enabling NOMA-OMA switching can further enhance the system performance while significantly increasing the complexity of the optimization task. In this study, we propose an optimized framework to jointly utilize the power domain NOMA in a vehicular network, where dynamic NOMA-OMA switching is enabled, by integrating the optimization of vehicle-to-RSU association, channel assignment, NOMA-OMA switching, and transmit power allocation into a single solution. To handle the complexity of these operations, we also propose simplified formulations that make the solution practical for real-time applications. The proposed framework reduces total power consumption by up to $27\%$ compared to Util&LB/opt, improves fairness in user association by $18\%$, and operates efficiently with minimal computational overhead. These findings highlight the potential of the proposed framework to enhance communication performance in dynamic, densely populated urban environments.

Intelligent manufacturing promises to revolutionize production processes, and it is expected to enhance efficiency and productivity. In this paper, we study the job-shop scheduling problem, which is a key enabler to the realization of intelligent manufacturing. Compared to previous studies, the proposed solution tackles generalized configurations by allowing the repetition of jobs and by assuming the limited capacity of machines and a sequence of operations constituting a job. The proposed solution is fully controlled by the trained reinforcement learning agent that finds the optimal match between the job and machine to schedule. In addition, by introducing the expected tardiness (ETD) metric, the agent can enhance the scheduling performance while effectively handling dynamic action space changes with the action masking technique. To effectively adapt to the particular manufacturing site, we propose a novel training approach that utilizes the average order arrival distribution learned from the historical logs. Such data-driven optimization can train an agent that effectively captures the general and site-specific characteristics of job arrivals, leading to improved generalization performance and a finely tuned model, respectively. To validate the proposed approach, we implement a custom environment with which extensive performance evaluation and comparison are carried out. The evaluation results show that the proposed approach can outperform the conventional heuristic priority dispatching rules under the desired performance criteria such as total tardiness and the total manufacturing cost. To be specific, in terms of the total cost metric, the proposed approach outperforms the considered approaches by 31.69% on average. In addition, the use of ETD can enhance the performance of the conventional approaches by 27.74% on average.

In high-density network environments with multiple access points (APs) and stations, individual uplink scheduling by each AP can severely interfere with the uplink transmissions of neighboring APs and their associated stations. In congested areas where concurrent uplink transmissions may lead to significant interference, it would be beneficial to deploy a cooperative scheduler or a central coordinating entity responsible for orchestrating cooperative uplink scheduling by assigning several neighboring APs to support the uplink transmission of a single station within a proximate service area to alleviate the excessive interference. Cooperative uplink scheduling facilitated by cooperative information sharing and management is poised to improve the likelihood of successful uplink transmissions in areas with a high concentration of APs and stations. Nonetheless, it is crucial to account for the queue stability of the stations and the potential delays arising from information exchange and the decision-making process in uplink scheduling to maintain the overall effectiveness of the cooperative approach. In this paper, we propose a Lyapunov drift-plus-penalty framework-based cooperative uplink scheduling method for densely populated Wi-Fi networks. The cooperative scheduler aggregates information, such as signal-to-interference-plus-noise ratio (SINR) and queue status. During the aggregation procedure, propagation delays are also estimated and utilized as a value of expected cooperation delays in scheduling decisions. Upon aggregating the information, the cooperative scheduler calculates the Lyapunov drift-plus-penalty value, incorporating a predefined model parameter to adjust the system accordingly. Among the possible scheduling candidates, the proposed method proceeds to make uplink decisions that aim to reduce the upper bound of the Lyapunov drift-plus-penalty value, thereby improving the network performance and stability without a severe increase in cooperation delay in highly congested areas. Through comprehensive performance evaluations, the proposed method effectively enhances network performance with an appropriate model parameter. The performance improvement is particularly notable in highly congested areas and is achieved without a severe increase in cooperation delays.