25 April 2019 in IEEE Internet of Things Journal
Recently, as the development of artificial intelligence (AI), data-driven AI methods have shown amazing performance in solving complex problems to support the Internet of Things (IoT) world with massive resource-consuming and delay-sensitive services. In this paper, we propose an intelligent Resource Allocation Framework (iRAF) to solve the complex resource allocation problem for the Collaborative Mobile Edge Computing (CoMEC) network. The core of iRAF is a multi-task deep reinforcement learning algorithm for making resource allocation decisions based on network states and task characteristics, such as the computing capability of edge servers and devices, communication channel quality, resource utilization, and latency requirement of the services, etc. The proposed iRAF can automatically learn the network environment and generate resource allocation decision to maximize the performance over latency and power consumption with self-play training. iRAF becomes its own teacher: a Deep Neural Network (DNN) is trained to predict iRAF’s resource allocation action in a self-supervised learning manner, where the training data is generated from the searching process of Monte Carlo Tree Search (MCTS) algorithm. A major advantage of MCTS is that it will simulate trajectories into the future, starting from a root state, to obtain a best action by evaluating the reward value. Numerical results show that our proposed iRAF achieves 59.27% and 51.71% improvement on service latency performance compared with the greedy-search and the deep Q-learning based methods, respectively.
20 July 2020 in IEEE Transactions on Vehicular Technology
Hybrid analog/digital processing is crucial for millimeter-wave (mmWave) MIMO systems due to its ability to balance the gain and cost. Despite fruitful recent studies, the optimal beamforming/combining method remains unknown for a practical multiuser, broadband mmWave MIMO equipped with low-resolution phase shifters and low-resolution analog-to-digital converters (ADCs). In this paper, we leverage artificial intelligence techniques to tackle this problem. Particularly, we propose a neural hybrid beamforming/combining (NHB) MIMO system, where the various types of hybrid analog/digital mmWave MIMO systems are transformed into a corresponding autoencoder (AE) based neural networks. Consequently, the digital and analog beamforming/combiners are obtained by training the AE based new model in an unsupervised learning manner, regardless of particular configurations. Using this approach, we can apply {a machine} learning-based design methodology that is compatible with a range of different beamforming/combing architectures. We also propose an iterative training strategy for the neural network parameter updating, which can effectively guarantee fast convergence of the established NHB model. According to simulation results, the proposed NHB can offer a significant performance gain over existing methods in terms of bit error rate (BER). Moreover, NHB can fast formulate the neural network parameters as channel changed, which is believed more promising for practice due to its better flexibility and compatibility.
11 September 2019 in IEEE Transactions on Signal Processing
Antenna selection is a promising technology to reduce the hardware complexity for massive multiple-input multiple-output (MIMO) system. However, the design of a near-optimal antenna selection algorithm with low searching complexity is still a challenge. In this paper, we describe a self-supervised learning Monte Carlo Tree Search (MCTS) method to solve the problem of selecting an antenna for the massive MIMO system. The search process for selecting antennas with maximal channel capacity is converted to a decision-making based problem. Based on the system model of antenna selection, we map the components of MIMO system to the basic elements of MCTS such as action, tree state, and reward. The three main search steps of MCTS--selecting, expanding, and backing up--are also redesigned for the antenna selection problem. To improve the search efficiency of the MCTS, we used a linear regression module to extract the features from the channel state information (CSI) and output the prediction to MCTS as prior probability. Since the data and label are generated by the MCTS process itself, the entire process can be considered as a self-supervised learning process. According to the simulation results, the proposed self-supervised learning MCTS-based antenna search method exhibits a high searching efficiency with near-optimal performance, which archives 40% and 15% outage capacity compared with random selection and greedy search selection, respectively. The bit-error rate (BER) performance of the proposed method has about 1-dB gain compared to the greedy search selection method. Compared with the state-of-the-art capacity optimal antenna search methods, the proposed self-supervised learning MCTS-based algorithm reduces about 50% search complexity.
22 August 2022 in IEEE Transactions on Vehicular Technology
Due to the maneuverability and flexibility of unmanned aerial vehicles (UAVs), the UAV-enabled data collection systems for wireless sensor networks (WSN) have received widespread attention. However, the state-of-the-art UAV trajectory designs mainly focus on static environments, which are not applicable in the practical scenarios considered in this work, e.g., mobile nodes, decommissioning of existing nodes, and new emergency nodes. This paper proposes a two-level deep reinforcement learning (DRL) framework to solve this challenge. The first-level deep neural network (DNN) is applied to model the dynamic changing environment. In the second level, we employ a deep Q-learning network to plan a trajectory online according to the environment features from the first level DNN. Besides, online trajectory planning is performed by a low-power UAV edge computing platform. To enable online planning on the power-constraint UAV edge-computing platform, all networks adopt a lightweight low-complexity optimization design. According to simulation results, the proposed system achieves higher data acquisition success rates when compared to existing state-of-the-art methods. We also perform field tests on the proposed UAV edge computing platform, which also demonstrates high data acquisition performance.
30 April 2020 in IEEE Transactions on Vehicular Technology
Autonomous driving has attracted significant attention in recent years. With the booming of artificial intelligence (AI), deep learning technologies have been applied to autonomous driving to help vehicles better perceive the environment. Besides the perceiving environment, predictive driving is another prominent smooth control and safe driving skill for human drivers. In this work, we develop a deep Monte Carlo Tree Search (deep-MCTS) control method for vision-based autonomous driving. Compared with existing deep learning-based autonomous driving control methods, our method can predict driving maneuvers to help improve the stability and performance of driving control. Two deep neural networks (DNNs) are employed for predicting action-state transformation and obtaining action-selection probabilities, respectively. The deep-MCTS utilizes the predicted information of the two DNNs and reconstructs multiple possible trajectories to predict driving maneuvers. An optimal trajectory is selected by the deep-MCTS based on both current road conditions and predicted driving maneuvers. The proposed method achieves high control stability by avoiding sharp turns and driving deviations. We implement our algorithm in the Udacity and Torcs self-driving environments. The testing results show that our algorithm achieves a significant improvement in training efficiency, the stability of steering control, and stability of driving trajectory compared to existing methods.
5 July 2017 in IEEE Transactions on Vehicular Technology
In this paper, we design and implement a long-range and broadband aerial communication system using directional antennas (ACDA). The system integrates Wi-Fi devices with aerial networks to quickly establish a Wi-Fi infrastructure in the air, which provides real-time communication capability for disasters where a fixed communication infrastructure does not exist. ACDA utilizes unmanned aerial vehicle (UAV)-carried directional antennas to extend communication range, increase throughput, and reduce interference. A GPS-based control algorithm is designed and implemented to automatically reject wind disturbance and align the directions of antennas in accordance with UAV movement. A received signal strength indicator-based decentralized initial scan algorithm is also designed and implemented to quickly establish initial connection between the UAVs. Simulation studies verify the effectiveness of the heading control and initial scan algorithm. Field tests are also conducted to evaluate the performance of overall system in terms of throughput and delay with respect to the increase of communication range. The ACDA prototype system achieves 48 Mb/s throughput at a distance of 300 m and 2 Mb/s at 5000 m, and proves the promising usage of directional antennas for long-distance Wi-Fi aerial communication. Practical use of this on-demand communication system to aid emergency response is also demonstrated through a case study in a real disaster drill.