Recently, unmanned aerial vehicles (UAVs) as flying wireless communication platform have attracted much attention. Benefiting from the mobility, UAV aerial base stations can be deployed quickly and flexibly, and can effectively establish Line-of-Sight communication links. However, there are many challenges in UAV communication system. The first challenge is energy constraint, where the UAV battery lifetime is in the order of fraction of an hour. The second challenge is that the coverage area of UAV aerial base station is limited and the commercial UAV is usually expensive. Thus, covering a large target region all the time with sufficient UAVs is quite challenging. To solve above challenges, in this paper, we propose energy efficient and fair 3-D UAV scheduling with energy replenishment, where UAVs move around to serve users and recharge timely to replenish energy. Inspired by the success of deep reinforcement learning, we propose a UAV Control policy based on Deep Deterministic Policy Gradient (UC-DDPG) to address the combination problem of 3-D mobility of multiple UAVs and energy replenishment scheduling, which ensures energy efficient and fair coverage of each user in a large region and maintains the persistent service. Simulation results reveal that UC-DDPG shows a good convergence and outperforms other scheduling algorithms in terms of data volume, energy efficiency and fairness.

Unmanned aerial vehicle (UAV) data collection is a promising research direction that can be applied to many practical scenarios. Due to the limited energy of sensors in wireless sensor networks (WSN), UAV, which is considered as a mobile fusion center, can effectively prolong the lifetime of sensor via supporting communication with the sensor directly. However, since the UAV’s energy constrained, it is necessary that multiple UAVs provide data collection to sensors in large areas. In this paper, we consider a scenario where multiple UAVs collect data from a set of two-dimensional distributed sensors. The UAV can communicate with sensors while flying or hovering. The goal is to minimize the total time that all UAVs from data center and return to data center after completing all collection tasks, while giving each sensor a certain amount of data and energy. The problem of minimizing the task completion time of multiple UAVs is still a big challenge. We solve the multi-UAV problem by jointly optimizing UAV-sensor association mechanism and data collection method of UAVs. The numerical results show that the proposed multi-UAV data collection scheme shortens the task completion time.

Traffic control is regarded as a key issue to alleviate congestion in internet of vehicles (IoV) machine-type communications (MTC). Recently, many traffic control schemes have been studied, such as access class barring (ACB) scheme and back-off (BO) scheme. However, the dynamics of traffic and the heterogeneous requirements of different IoV applications are not considered in most existing studies, which is significant for the random access resource allocation. In this paper, we consider a hybrid scheme, combining the priority dynamic ACB (PDACB) scheme and BO scheme. The IoV devices are classified depending on different delay characteristics, where the delay-sensitive devices are classified as high priority. The target is to maximum the successful transmission of packets with the success rate constraint by adjusting the various ACB factors. Proximal policy optimization (PPO) algorithm as a unique deep reinforcement learning (DRL) method is utilized in this paper, which can obtain continuous action space and solve for the optimal ACB factors without estimating backlog of nodes. A quick convergence is achieved by designing sensible state space, action space and reward. The access capability of the PDACB traffic control scheme is verified by simulations.

In this letter, we present Wi-Mose, the first 3D moving human pose estimation system using commodity WiFi. Previous WiFi-based works have achieved 2D and 3D pose estimation. These solutions either capture poses from one perspective or construct poses of people who are at a fixed point, preventing their wide adoption in daily scenarios. To reconstruct 3D poses of people who move throughout the space, we fuse the amplitude and phase of Channel State Information (CSI) into CSI image which can provide both pose and position information. Besides, we design a neural network to extract features which are only associated with poses from CSI images and then convert the features into key-point coordinates. Experimental results show that Wi-Mose can localize key-point with 29.7 mm and 37.8 mm Procrustes analysis Mean Per Joint Position Error (P-MPJPE) in the Line of Sight (LoS) and Non-Line of Sight (NLoS) scenarios, respectively, achieving higher performance than the state-of-the-art method. The results indicate that Wi-Mose can capture high-precision 3D human poses throughout the space.

The continuous development of intelligent traffic control systems has a profound influence on urban traffic planning and traffic management. Indeed, as big data and artificial intelligence continue to evolve, the traffic control strategy based on deep reinforcement learning (RL) has been proven to be a promising method to improve the efficiency of intersections and save people’s travel time. However, the existing algorithms ignore the temporal and spatial characteristics of intersections. In this article, we propose a multiagent RL based on the deep spatiotemporal attentive neural network (MARL-DSTAN) to determine the traffic signal timing in a large-scale road network. In this model, the state information captures the spatial dependency of the entire road network by leveraging the graph convolutional network (GCN) and integrates the information based on the importance of intersections via the attention mechanism. Meanwhile, to accumulate more valuable samples and enhance the learning efficiency, the recurrent neural network (RNN) is introduced in the exploration stage to constrain the action search space instead of fully random exploration. MARL-DSTAN decomposes the large-scale area into multiple base environments, and the agents in each base environment use the idea of “centralized training and decentralized execution” to learn to accelerate the algorithm convergence. The simulation results show that our algorithm significantly outperforms the fixed timing scheme and several other state-of-the-art baseline RL algorithms.

Human sensing based on commodity Wi-Fi devices has become a promising technique in human tracking, gesture recognition, walking speed monitoring, in-home healthcare, etc. However, past human sensing systems usingWi-Fi capture limited information about humans. Hence in this letter, we try to make commodity Wi-Fi devices act as cameras to directly capture human poses, i.e., fine-grained human skeleton images. We use a synchronized camera to capture human skeletons as annotations for Wi-Fi signals and design a novel neural network to convert Wi-Fi signals into images. We utilize three transceivers coordinately and use amplitude and phase information of Channel State Information (CSI) jointly to improve the resolution of Wi-Fi signals. We also introduce a method to extract useful and accurate CSI corresponding to humans and construct CSI images which are input of the neural network. Experimental results show that commodity Wi-Fi devices can capture human poses almost as fine-grained as cameras.