25 August 2020 in IEEE Journal on Selected Areas in Communications
In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers. Specifically, IoT devices contend for accessing the shared wireless channel using an adaptive $p$ -persistent slotted Aloha protocol; and the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to decode multiple received data from IoT devices to improve access efficiency. To enable an energy-sustainable capacity-optimal network, we study the joint problem of dynamic multi-UAV altitude control and multi-cell wireless channel access management of IoT devices as a stochastic control problem with multiple energy constraints. We first formulate this problem as a Constrained Markov Decision Process (CMDP), and propose an online model-free Constrained Deep Reinforcement Learning (CDRL) algorithm based on Lagrangian primal-dual policy optimization to solve the CMDP. Extensive simulations demonstrate that our proposed algorithm learns a cooperative policy in which the altitude of UAVs and channel access probability of IoT devices are dynamically controlled to attain the maximal long-term network capacity while ensuring energy sustainability of UAVs, outperforming baseline schemes. The proposed CDRL agent can be trained on a small network, yet the learned policy can efficiently manage networks with a massive number of IoT devices and varying initial states, which can amortize the cost of training the CDRL agent.
9 November 2017 in IEEE Communications Surveys & Tutorials
With the proliferation of wireless devices, wireless networks in various forms have become global information infrastructure and an important part of our daily life, which, at the same time, incur fast escalations of both data volumes and energy demand. In other words, energy-efficient wireless networking is a critical and challenging issue in the big data era. In this paper, we provide a comprehensive survey of recent developments on energy-efficient wireless networking technologies that are effective or promisingly effective in addressing the challenges raised by big data. We categorize existing research into two main parts depending on the roles of big data. The first part focuses on energy-efficient wireless networking techniques in dealing with big data and covers studies in big data acquisition, communication, storage, and computation; while the second part investigates recent approaches based on big data analytics that are promising to enhance energy efficiency of wireless networks. In addition, we identify a number of open issues and discuss future research directions for enhancing energy efficiency of wireless networks in the big data era.
13 May 2019 in IEEE Transactions on Vehicular Technology
The recent explosive adoption of electric vehicles (EVs) and plug-in hybrid electric vehicles (PHEVs) has sparked a considerable interest in academia in developing efficient charging schemes. Supported by the advanced vehicle-to-grid (V2G) network, vehicles and charging stations can respectively make better charging and pricing decisions via real-time information sharing. In this paper, we study the charging problem in an intelligent transportation system, which consists of smart-grid enabled charging stations and networked EVs. Each vehicle aims to select a station with the lowest charging cost by considering the charging prices and its location while the objective of a charging station is to maximize its revenue given the charging strategy of the vehicles. We employ a multi-leader multi-follower Stackelberg game to model the interplay between the vehicles and charging stations, in which the location factor plays an important role. We show that the equilibrium of the followers' subgame played by the vehicles exits, while the stations are able to reach an equilibrium of their subgame with respect to the charging prices. Therefore, the Nash equilibrium of the Stackelberg game is achievable through the proposed charging scheme. Finally, the performance of the proposed approach is demonstrated via extensive trace-driven simulation.
21 May 2020 in IEEE Internet of Things Journal
Recent advances in communications technologies and the proliferation of connected devices have spawned a variety of information-centric Internet of Things (IoT) systems where timely information updating is normally required. Age of Information (AoI) has recently been introduced to quantify the freshness of the knowledge the controller has about the remote information sources. With the development of IoT applications, it is becoming increasingly common that the application-level services, e.g., status updates, rely on the timely delivery of fresh information from a number of information sources. In this paper, we study the application-oriented scheduling for optimizing information freshness in the presence of correlated information sources. To this end, we adopt the concept of Age of Correlated Information (AoCI) and formulate the scheduling problem as an episodic Markov decision process (MDP) problem with an application-oriented policy. Given the complexity of the above problem, we develop a learning-based approach that not only leverages the emerging deep reinforcement learning (DRL) techniques but also exploits diverse domain knowledge. Numerical results show that the proposed approach achieves better performance in terms of AoCI, compared to some typical baseline methods.
17 April 2018 in IEEE Transactions on Network Science and Engineering
With the superior capability of discovering intricate structure of large data sets, deep learning has been widely applied in various areas including wireless networking. While existing deep learning applications mainly focus on data analysis, the role it can play on fundamental research issues in wireless networks is yet to be explored. With the proliferation of wireless networking infrastructure and mobile applications, wireless network optimization has seen a tremendous increase in problem size and complexity, calling for a paradigm for efficient computation. This paper presents a pioneering study on how to exploit deep learning for significant performance gain in wireless network optimization. Analysis on the flow constrained optimization problems suggests the possibility that a smaller-sized problem can be solved while sharing equally optimal solutions with the original problem, by excluding the potentially unused links from the problem formulation. To this end, we design a deep learning framework to find the latent relationship between flow information and link usage by learning from past computation experience. Numerical results demonstrate that the proposed method is capable of identifying critical links and can reduce computation cost by up to 50% without affecting optimality, thus greatly improve the efficiency of solving network optimization problems.
6 July 2018 in IEEE Internet of Things Journal
Wireless network resource allocation is an important issue for designing Internet of Things systems. In this paper, we consider the problem of wireless network capacity optimization that involves issues such as flow allocation, link scheduling and power control. We show that it can be decomposed into a linear program and a nonlinear weighted sum-rate maximization problem for power allocation. Unlike most traditional methods that iteratively search the optimal solutions of the nonlinear subproblem, we propose to directly compute approximated solutions based on machine learning techniques. Specifically, the learning systems consist of both support vector machines (SVMs) and deep belief networks (DBNs) that are trained based on offline computed optimal solutions. In the running phase, the SVMs perform classification for each link to decide whether to use maximal transmit power or be turned off. At the same time, the DBNs compute an approximation of the optimal power allocation. The two results are combined to obtain an approximated solution of the nonlinear program. Simulation results demonstrate the effectiveness of the proposed machine learning based algorithm.