Recently, unmanned aerial vehicles (UAVs) as flying wireless communication platform have attracted much attention. Benefiting from the mobility, UAV aerial base stations can be deployed quickly and flexibly, and can effectively establish Line-of-Sight communication links. However, there are many challenges in UAV communication system. The first challenge is energy constraint, where the UAV battery lifetime is in the order of fraction of an hour. The second challenge is that the coverage area of UAV aerial base station is limited and the commercial UAV is usually expensive. Thus, covering a large target region all the time with sufficient UAVs is quite challenging. To solve above challenges, in this paper, we propose energy efficient and fair 3-D UAV scheduling with energy replenishment, where UAVs move around to serve users and recharge timely to replenish energy. Inspired by the success of deep reinforcement learning, we propose a UAV Control policy based on Deep Deterministic Policy Gradient (UC-DDPG) to address the combination problem of 3-D mobility of multiple UAVs and energy replenishment scheduling, which ensures energy efficient and fair coverage of each user in a large region and maintains the persistent service. Simulation results reveal that UC-DDPG shows a good convergence and outperforms other scheduling algorithms in terms of data volume, energy efficiency and fairness.

Unmanned aerial vehicle (UAV) data collection is a promising research direction that can be applied to many practical scenarios. Due to the limited energy of sensors in wireless sensor networks (WSN), UAV, which is considered as a mobile fusion center, can effectively prolong the lifetime of sensor via supporting communication with the sensor directly. However, since the UAV’s energy constrained, it is necessary that multiple UAVs provide data collection to sensors in large areas. In this paper, we consider a scenario where multiple UAVs collect data from a set of two-dimensional distributed sensors. The UAV can communicate with sensors while flying or hovering. The goal is to minimize the total time that all UAVs from data center and return to data center after completing all collection tasks, while giving each sensor a certain amount of data and energy. The problem of minimizing the task completion time of multiple UAVs is still a big challenge. We solve the multi-UAV problem by jointly optimizing UAV-sensor association mechanism and data collection method of UAVs. The numerical results show that the proposed multi-UAV data collection scheme shortens the task completion time.

To solve the policy optimizing problem in many scenarios of smart wireless network management using a single universal algorithm, this letter proposes a universal learning framework which is called AI Framework based on Deep Reinforcement Learning (DRL). This framework can also solve the problem that the state is painful to design in traditional Reinforcement Learning (RL). This AI Framework adopts Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) to model the potential spatial features (i.e. location information) and sequential features from the raw wireless signal automatically. These features can be taken as the state definition of DRL. Meanwhile, this framework is suitable for many scenarios such as resource management and access control due to DRL. The mean value of throughput, the standard deviation of throughput and handover counts are used to evaluate its performance on the mobility management problem in Wireless Local Area Network (WLAN) on a practical testbed. The results show that the framework gets significant improvements and learns intuitive features automatically.

The continuous development of intelligent traffic control systems has a profound influence on urban traffic planning and traffic management. Indeed, as big data and artificial intelligence continue to evolve, the traffic control strategy based on deep reinforcement learning (RL) has been proven to be a promising method to improve the efficiency of intersections and save people’s travel time. However, the existing algorithms ignore the temporal and spatial characteristics of intersections. In this article, we propose a multiagent RL based on the deep spatiotemporal attentive neural network (MARL-DSTAN) to determine the traffic signal timing in a large-scale road network. In this model, the state information captures the spatial dependency of the entire road network by leveraging the graph convolutional network (GCN) and integrates the information based on the importance of intersections via the attention mechanism. Meanwhile, to accumulate more valuable samples and enhance the learning efficiency, the recurrent neural network (RNN) is introduced in the exploration stage to constrain the action search space instead of fully random exploration. MARL-DSTAN decomposes the large-scale area into multiple base environments, and the agents in each base environment use the idea of “centralized training and decentralized execution” to learn to accelerate the algorithm convergence. The simulation results show that our algorithm significantly outperforms the fixed timing scheme and several other state-of-the-art baseline RL algorithms.

Rate adaptation (RA) is an essential mechanism in 802.11 WLAN. In the latest 802.11ac protocol, there are two emerging problems that need to be addressed for RA. Firstly, 802.11ac supporting more rate selections requires more efforts to find the optimal rate. Secondly, 802.11ac supports higher rate. The difference between optimal rate and non-optimal rate can be so great that non-optimal rate would severely deteriorate the throughput of the WLAN. In order to tackle these problems, we develop a novel RA algorithm termed Rate Adaptation with Thompson Sampling (RATS) for stationary and non-stationary channel environments. In this algorithm, we first consider compacting the search space by removing some rates to accelerate the convergence of the algorithm. Moreover, inspired by multi-armed bandit problem, we design RA algorithm based on Thompson Sampling. Simulation results demonstrate that the performance of the proposed RATS outperforms the existing method.

In order to meet the ever-increasing traffic demand of Wireless Local Area Networks (WLANs), channel bonding is introduced in IEEE 802.11 standards. Although channel bonding effectively increases the transmission rate, the wider channel reduces the number of non-overlapping channels and is more susceptible to interference. Meanwhile, the traffic load differs from one access point (AP) to another and changes significantly depending on the time of day. Therefore, the primary channel and channel bonding bandwidth should be carefully selected to meet traffic demand and guarantee the performance gain. In this paper, we proposed an On-Demand Channel Bonding (O-DCB) algorithm based on Deep Reinforcement Learning (DRL) for heterogeneous WLANs to reduce transmission delay, where the APs have different channel bonding capabilities. In this problem, the state space is continuous and the action space is discrete. However, the size of action space increases exponentially with the number of APs by using single-agent DRL, which severely affects the learning rate. To accelerate learning, Multi-Agent Deep Deterministic Policy Gradient (MADDPG) is used to train O-DCB. Real traffic traces collected from a campus WLAN are used to train and test O-DCB. Simulation results reveal that the proposed algorithm has good convergence and lower delay than other algorithms.