After decades of unprecedented development, modern networks have evolved far beyond expectations in terms of scale and complexity. In many cases, traditional traffic engineering (TE) approaches fail to address the quality of service (QoS) requirements of modern networks. In recent years, deep reinforcement learning (DRL) has proved to be a feasible and effective solution for autonomously controlling and managing complex systems. Massive growth in the use of DRL applications in various domains is beginning to benefit the communications industry. In this paper, we firstly provide a comprehensive overview of DRL-based TE. Then, we present a detailed literature review on applications of DRL for TE including three fundamental issues: routing optimization, congestion control, and resource management. Finally, we discuss our insights into the challenges and future research perspectives of DRL-based TE.

As a promising technology in the 5G era, the artificial intelligence (AI) enabled Internet of controllable things (IoCT) is expected to be an integral part of heterogeneous networks (HetNets) in the future. However, the realization of ultra-reliable low-latency communications (URLLC) in IoCT communications underlaid HetNet has stringent quality of service (QoS) requirements, resulting in unprecedented challenges for existing wireless resource allocation methods. In this paper, we first describe a cellular HetNet model with uplink IoCT communications, then formulate a dynamic mixed-integer nonlinear programming (MINLP) resource allocation problem for maximizing the long-term average energy efficiency under URLLC requirements including reliability, latency, and transmission rate. To solve the problem, we propose a decentralized MADRL-based resource allocation algorithm with a decentralized partially observable Markov decision process (dec-POMDP) and a mixed-centralized-decentralized (MCD) framework to address the partial observability and the scalability issues, respectively. In addition, we design a reward function featuring the objective decomposition, baseline-guided scaling, and QoS violation penalty so that the agents are coordinated. Extensive experiments demonstrate the convergence, scalability, and robustness of the proposed algorithm. Besides, the proposed algorithm substantially outperforms conventional resource allocation methods and different agent communication mechanisms in terms of maximizing energy efficiency.

With the increasing cases of thyroid malignant tumor, the diagnosis of thyroid nodule has attracted more and more attention. Deep learning has achieved promising results in computer-aided diagnosis due to the advantages of obtaining high-dimensional features. In this paper, we proposed hybrid multi-branch convolutional neural network based on feature cropping method for feature extraction and classification of thyroid nodule ultrasound images. Firstly, we designed backbone convolutional neural network to extract shared feature maps and classification network as global branch. Next, we added a feature cropping branch in the network to perform multi-cropping on batch feature maps, so as to reduce the impact on classification caused by the similarity of local features between benign and malignant thyroid nodule images. Finally, based on softmax predictions of different branch feature maps, we utilize a weighted cross-entropy loss function to train our proposed binary-classification network. Experimental results show that our proposed method has achieved 96.13% accuracy, 93.24% precision, 97.18% recall, and 95.17% F1-measure in public dataset and local dataset, outperforming other models.

Diffusion-weighted magnetic resonance imaging (DWI) is sensitive to acute ischemic stroke and is a common diagnostic method for the stroke. However, the diagnostic result relies on the visual observation of neurologists which may vary from doctor to doctor under different circumstance. And manual segmentation is often a time-consuming and subjective process. The time from onset to thrombus removal has a significant impact on the prognosis of patients with acute ischemic stroke. The shorter the time, the better the prognosis. For this purpose we present a novel framework to quickly and automatically segment the ischemic stroke lesions on DWI. We mainly have three contributions: firstly, we design a detection and segmentation network (DSN) to solve the two kinds of data imbalance; secondly, we propose a triple-branch DSN architecture, used for extracting different plane feature respectively; thirdly, we propose a multi-plane fusion network (MPFN), which aims to make final prediction more accurate. Extensive experiments on ISLES2015 SSIS DWI sequence dataset demonstrate the superiority of our proposed segmentation method. The dice reached 62.2% and the sensitivity reached 71.7%.

Mobile edge computing (MEC) is an enabling technology for next-generation network architectures to deliver more diversified communication services and meet more demanding quality-of-service (QoS) requirements. However, owing to the growing scale and heterogeneity of the networks, energy-efficient resource allocation in heterogeneous MEC (Het-MEC) networks faces great challenges. As an emerging research area, multi-agent deep reinforcement learning (MADRL) is expected to realize autonomous resource allocation in Het-MEC networks by learning from trial and error. Nevertheless, existing MADRL-based solutions are usually not applicable to practical scenarios by the limitations of centralized control schemes or massive signaling overhead. To address the issue, we first formulate the energy-efficient resource allocation problem in Het-MEC networks as a time-variant mixed-integer nonlinear programming (MINLP) problem. Then, we propose a fully decentralized collaborative MADRL-based algorithm featuring the multi-actor shared-critic (MASC) architecture and the regional training distributed execution (RTDE) framework, which effectively stabilizes model training and reduces information exchange, respectively. Finally, we conduct extensive simulations to evaluate the performance of the proposed algorithm. Numerical results demonstrate that the proposed algorithm comprehensively outperforms several mainstream baseline methods in terms of convergence performance, scalability, and robustness.

This letter investigates a deep reinforcement learning (DRL)-based spectrum access scheme for device-to-device (D2D) communication underlay cellular networks. Specifically, cellular users (CUEs) and D2D pairs attempt to access the time slots (TSs) of a shared spectrum, and TSs are dynamically scheduled to CUEs in different frames. Based on the DRL theory, D2D pairs can be seen as a centralized agent which aims to learn an optimal spectrum access strategy to maximize the sum throughput without any prior information. In particular, with different locations of CUEs, the spectrum access manners for D2D communication are changed to ensure the communication quality of CUEs at the cell edge. Then, a double deep Q-network (DDQN) based D2D spectrum access (D ⁴ SA) algorithm is proposed, which makes D2D pairs learn to decide whether to access the spectrum in different TSs. Moreover, to ensure the fairness of resource allocation among D2D pairs, we improve the proposed algorithm and incorporate fairness into the objective function. Simulation results show that our proposed algorithm can achieve an optimal sum throughput close to the theoretical upper bound, where the performance is significantly improved compared to the scheme based on base station cooperation.