Mobile edge computing offloads compute-intensive tasks generated on mobile wireless devices (WD) to edge servers (ES), which provides mobile users with low-latency computing services. Opportunistic computing offloading is effective to enhance computing performance in dynamic edge network environments; however, careless offloading of tasks to ESs can lead to WDs preempting network computing resources with limited bandwidth, thereby resulting in inefficient allocation of computing resources. To address these challenges, this paper proposes the density clustering and ensemble learning training–based deep reinforcement learning (DCEDRL) method for task offloading decision-making in mobile edge computing (MEC). Firstly, DCEDRL utilizes multiple deep neural networks to explore the environment. It trains multiple models using ensemble learning methods to obtain a combination of prediction results. Secondly, DCEDRL utilizes an optimized density clustering method to identify and classify computing tasks with similar characteristics to improve subsequent task scheduling and resource allocation efficiency. Finally, according to the stored priority information, DCEDRL utilizes the priority weight to resample the samples, adjust the sampling strategy in real time, and improve the adaptability and robustness of the system. Simulation results demonstrate that the proposed DCEDRL method can reduce the backlog of tasks by greater than over 21% compared to the baseline algorithms.

Due to the rapid development of network communication technology and the significant increase in network terminal equipment, the application of new network architecture software‐defined networking (SDN) combined with reinforcement learning in network traffic scheduling has become an important focus of research. Because of network traffic transmission variability and complexity, the traditional reinforcement‐learning algorithms in SDN face problems such as slow convergence rates and unbalanced loads. The problems seriously affect network performance, resulting in network link congestion and the low efficiency of inter‐stream bandwidth allocation. This paper proposes an automatic load‐balancing architecture based on reinforcement learning (ALBRL) in SDN. In this architecture, we design a load‐balancing optimization model in high‐load traffic scenarios and adapt the improved Deep Deterministic Policy Gradient (DDPG) algorithm to find a near‐optimal path between network hosts. The proposed ALBRL uses the sampling method of updating the experience pool with the SumTree structure to improve the random extraction strategy of the empirical‐playback mechanism in DDPG. It extracts a more meaningful experience for network updating with greater probability, which can effectively improve the convergence rate. The experiment results show that the proposed ALBRL has a faster training speed than existing reinforcement‐learning algorithms and significantly improves network throughput.

Optical transport networks (OTNs) are widely used in backbone- and metro-area transmission networks to increase network transmission capacity. In the OTN, it is particularly crucial to rationally allocate routes and maximize network capacities. By employing deep reinforcement learning (DRL)- and software-defined networking (SDN)-based solutions, the capacity of optical networks can be effectively increased. However, because most DRL-based routing optimization methods have low sample usage and difficulty in coping with sudden network connectivity changes, converging in software-defined OTN scenarios is challenging. Additionally, the generalization ability of these methods is weak. This paper proposes an ensembles- and message-passing neural-network-based Deep Q-Network (EMDQN) method for optical network routing optimization to address this problem. To effectively explore the environment and improve agent performance, the multiple EMDQN agents select actions based on the highest upper-confidence bounds. Furthermore, the EMDQN agent captures the network’s spatial feature information using a message passing neural network (MPNN)-based DRL policy network, which enables the DRL agent to have generalization capability. The experimental results show that the EMDQN algorithm proposed in this paper performs better in terms of convergence. EMDQN effectively improves the throughput rate and link utilization of optical networks and has better generalization capabilities.

Load-balancing optimization in software-defined networking (SDN) has been researched for a long time. Researchers have proposed many solutions to the load-balancing problem but have rarely considered the impact of transmission delay between controllers and switches under high-load network conditions. In this paper, we propose an adaptive load-balancing architecture based on link-state prediction (ALBLP) in SDN that can solve the influence of transmission delay between controllers and switches on network load balancing. ALBLP constructs the prediction model of the network link status, adopts the long-term and short-term memory neural network (LSTM) algorithm to predict the network link-state value, and then uses the predicted value as the Dijkstra weight to calculate the optimal path between network hosts. The proposed architecture can adaptively optimize network load balancing and avoid the empty window period, in which the switch flow table does not exist by actively issuing the flow table. Under the network architecture, we collect the data set of the network link-state by simulating the GÉANT network, and we verify the effectiveness of the proposed algorithm. The experiment results show that the ALBLP proposed in this paper can optimize load balancing in SDN and solve the problem of transmission delay between controllers and switches. It has a maximum load-balancing improvement of 23.7% and 11.7% in comparison with the traditional Open Shortest Path First (OSPF) algorithm and the reinforcement learning method based on Q-Learning.

Software-defined networking (SDN) has become one of the critical technologies for data center networks, as it can improve network performance from a global perspective using artificial intelligence algorithms. Due to the strong decision-making and generalization ability, deep reinforcement learning (DRL) has been used in SDN intelligent routing and scheduling mechanisms. However, traditional deep reinforcement learning algorithms present the problems of slow convergence rate and instability, resulting in poor network quality of service (QoS) for an extended period before convergence. Aiming at the above problems, we propose an automatic QoS architecture based on multistep DRL (AQMDRL) to optimize the QoS performance of SDN. AQMDRL uses a multistep approach to solve the overestimation and underestimation problems of the deep deterministic policy gradient (DDPG) algorithm. The multistep approach uses the maximum value of the n-step action currently estimated by the neural network instead of the one-step Q-value function, as it reduces the possibility of positive error generated by the Q-value function and can effectively improve convergence stability. In addition, we adapt a prioritized experience sampling based on SumTree binary trees to improve the convergence rate of the multistep DDPG algorithm. Our experiments show that the AQMDRL we proposed significantly improves the convergence performance and effectively reduces the network transmission delay of SDN over existing DRL algorithms.