Future-generation wireless networks (5G and beyond) must accommodate surging growth in mobile data traffic and support an increasingly high density of mobile users involving a variety of services and applications. Meanwhile, the networks become increasingly dense, heterogeneous, decentralized, and ad hoc in nature, and they encompass numerous and diverse network entities. Consequently, different objectives, such as high throughput and low latency, need to be achieved in terms of service, and resource allocation must be designed and optimized accordingly. However, considering the dynamics and uncertainty that inherently exist in wireless network environments, conventional approaches for service and resource management that require complete and perfect knowledge of the systems are inefficient or even inapplicable. Inspired by the success of machine learning in solving complicated control and decision-making problems, in this article we focus on deep reinforcement- learning (DRL)-based approaches that allow network entities to learn and build knowledge about the networks and thus make optimal decisions locally and independently. We first overview fundamental concepts of DRL and then review related works that use DRL to address various issues in 5G networks. Finally, we present an application of DRL for 5G network slicing optimization. The numerical results demonstrate that the proposed approach achieves superior performance compared with baseline solutions.

Driven by the emerging use cases in massive access future networks, technological advancements and evolutions are needed for wireless communications beyond the fifth-generation (5G) networks. In particular, we envisage that the upcoming sixth-generation (6G) networks with numerous devices will demand extremely high-performance interconnections, even under strenuous scenarios such as diverse mobility, extreme density, and dynamic environments. To cater to such a demand, an investigation of flexible and sustainable radio access network (RAN) techniques to support highly diverse requirements and massive connectivity is of utmost importance. To this end, this paper first presents the key driving applications for 6G, including smart cities and factories, unmanned aerial vehicles, and multi-dimensional sensing services, which necessitate the transformation of existing RAN techniques to achieve the key performance indicators required for 6G networks. In particular, we investigate the flexibility, massive interconnectivity, and energy efficiency aspects of 6G RANs and provide an in-depth discussion on the potential key enabling technologies. Last but not least, we present an artificial-intelligence-assisted network architecture for flexible 6G massive RANs with energy harvesting.

In this letter, we study the on-demand UAV-BS placement problem for arbitrarily distributed users. This UAV-BS placement problem is modeled as a knapsack-like problem, which is NP-complete. We propose a density-aware placement algorithm to maximize the number of covered users subject to the constraint of the minimum required data rates per user. Simulations are conducted to evaluate the performance of the proposed algorithm in a real environment with different user densities. Our numerical results indicate that for various user densities our proposed solution can service more users with guaranteed data rates compared to the existing method, while reducing the transmit power by 29%.

Multitenant cellular network slicing has been gaining huge interest recently. However, it is not well-explored under the heterogeneous cloud radio access network (H-CRAN) architecture. This paper proposes a dynamic network slicing scheme for multitenant H-CRANs, which takes into account tenants' priority, baseband resources, fronthaul and backhaul capacities, quality of service (QoS) and interference. The framework of the network slicing scheme consists of an upper-level, which manages admission control, user association and baseband resource allocation; and a lower-level, which performs radio resource allocation among users. Simulation results show that the proposed scheme can achieve a higher network throughput, fairness and QoS performance compared to several baseline schemes.

In this paper, we present an analytical framework to evaluate the latency performance of connection-based spectrum handoffs in cognitive radio (CR) networks. During the transmission period of a secondary connection, multiple interruptions from the primary users result in multiple spectrum handoffs and the need of predetermining a set of target channels for spectrum handoffs. To quantify the effects of channel obsolete issue on the target channel predetermination, we should consider the three key design features: 1) general service time distribution of the primary and secondary connections; 2) different operating channels in multiple handoffs; and 3) queuing delay due to channel contention from multiple secondary connections. To this end, we propose the preemptive resume priority (PRP) M/G/1 queuing network model to characterize the spectrum usage behaviors with all the three design features. This model aims to analyze the extended data delivery time of the secondary connections with proactively designed target channel sequences under various traffic arrival rates and service time distributions. These analytical results are applied to evaluate the latency performance of the connection-based spectrum handoff based on the target channel sequences mentioned in the IEEE 802.22 wireless regional area networks standard. Then, to reduce the extended data delivery time, a traffic-adaptive spectrum handoff is proposed, which changes the target channel sequence of spectrum handoffs based on traffic conditions. Compared to the existing target channel selection methods, this traffic-adaptive target channel selection approach can reduce the extended data transmission time by 35 percent, especially for the heavy traffic loads of the primary users.

Integrating unmanned aerial vehicles with RIS (UAV–RIS) can offer ubiquitous deployment services in communication-disabled areas, but is limited by the on-board energy of the UAVs. In this paper, a novel energy harvesting (EH) scheme on top of the UAV–RIS system, called EH-RIS scheme, is developed for the next generation high performance wireless system. The proposed EH-RIS scheme extends the simultaneous wireless information and power transfer (SWIPT) system by splitting the passive reflected arrays on the geometric space for transporting information and harvesting energy simultaneously. However, pedestrian mobility, and rapid channel changes post challenges to efficient resource allocation in wireless systems. Thus, a robust deep reinforcement learning (DRL)-based algorithm is developed to improve the proposed EH-RIS scheme for guaranteeing the quality of service (QoS) under dynamic wireless environments. The simulation results demonstrate the effectiveness and efficiency of the proposed robust DRL-based EH-RIS system, which not only outperform the existing state-of-the-art solutions but also approach to the performance of the exhaustive search method.