Electric Vehicles (EVs) have been encouraged to penetrate deeper in the vehicle market for the green transportation system. One of the key issues to promote EV industry is to deploy Battery Swapping Stations (BSSs) that can satisfy the electricity demand of EV users. Since large scale data of vehicles such as GPS locations and electricity requests can be collected, the data-driven approach can be a cost-effective and useful method to select the locations of BSSs. In this paper, we propose a data-driven framework to solve the BSS location selection problem based on a large scale of GPS data of taxies in metropolitan area. The proposed solution consists of three main steps: Hidden Markov Model (HMM) based map matching and trajectory extraction, electricity consumption rate model based battery swapping demand estimation and clustering strategy based BSS location determination. Compared to the state-of-art deployment baseline, our proposed scheme is more easily to implement in reality and the mean distance error between the location that battery swapping demand is generated and the nearest BSS is reduced by 52.5% and 62.7%, which will definitely reduce the range anxiety of EV users and help improve the will of using EVs.
21 September 2020 in IEEE Internet of Things Journal
Driven by the soaring traffic demand and the growing diversity of mobile services, wireless networks are evolving to be increasingly dense and heterogeneous. Accordingly, in such large-scale and complicated wireless networks, the optimal controlling is reaching unprecedented levels of complexity, while its traditional solutions of handcrafted offline algorithms become inefficient due to high complexity, low robustness, and high overhead. Therefore, reinforcement learning, which enables network entities to learn from their actions and consequences in the interactive network environment, attracts significant attentions. In this paper, we comprehensively review the applications of reinforcement learning in wireless networks from a layering perspective. First, we present an overview of the principle, fundamentals and several advanced models of reinforcement learning. Then we review the up-to-date applications of reinforcement learning in various functionality blocks of different network layers, ranging from the low-level physical layer, to the high-level application layer. Finally, we outline a broad spectrum of challenges, open issues, and future research directions of reinforcement learning empowered wireless networks.
26 June 2024 in IEEE Transactions on Vehicular Technology
Reconfigurable intelligent surface (RIS) is envisioned as a new technology to improve the quality-of-service in vehicular edge computing (VEC) networks due to its ability to change the wireless radio propagation environment. In this paper, we study multi-RIS-assisted VEC networks, where vehicle user equipments (VUEs) can offload tasks to nearby base stations (BSs) with integrated edge services (ESs) and select an efficient computation ES. Meanwhile, the individual virtual machine (VM), which is defined as a software clone of the VUE's service environment containing the profile and application to run the VUE's tasks, need to be migrated to the same ES to complete the process of the offloaded tasks. Accordingly, we formulate a throughput maximization for multi-RIS-assisted VEC networks via jointly optimizing the selected ESs for the VUEs, the deployment locations of RISs, and reflection matrices of RISs, subject to the maximum tolerable delay. To solve the non-convex mixed-integer nonlinear optimization problem, we propose an efficient algorithm based on the multi-agent proximal policy optimization (MAPPO) with the centralized training and decentralized execution (CTDE) framework, where both the BSs with integrated ESs and the VUEs are regarded as two types of heterogeneous agents interacting with the environment. In particular, several tricks such as reward normalization, orthogonal initialization, and learning rate decay are adopted to improve the convergence performance of the MAPPO-based algorithm. Numerical simulation results demonstrate the fast convergence behavior of the proposed MAPPO-based algorithm compared to other conventional multi-agent reinforcement learning (MARL) algorithms. Moreover, the results also show that the throughput obtained by the proposed MAPPO-based scheme outperforms that obtained by the scheme without multi-RIS 26.41% and that obtained by the scheme without service migration 23.65%, respectively. Besides, it is verified that deploying locations of RISs is a valid solution to enhance the throughput of VEC networks.
29 September 2021 in IEEE Transactions on Vehicular Technology
The explosively increasing traffic demand in the fifth generation (5 G) mobile communication system has made the limited licensed spectrum quite competitive. Such status quo motivates the development of new radio (NR) based access to the unlicensed band, which is a promising solution for system performance enhancement. In this paper, we focus on the coexistence between the NR system and the WiFi system based on the duty cycle mechanism. Firstly, an optimization model is formulated to maximize the throughput of the NR system in the unlicensed band (NR-U) via the joint bandwidth and transmission opportunity allocation under the fairness requirement. Then, to solve this optimization problem, we convert it into a Markov decision process with continuous action space, based on which a deep deterministic policy gradient (DDPG) based coexistence algorithm is proposed. At last, simulation results show that the proposed algorithm can ensure the coexistence between the NR-U system and the WiFi system from three aspects. Firstly, the throughput improvements of the NR-U system under different fairness thresholds and different number of user equipments and access points are illustrated. Secondly, the proposed algorithm adapts to the dynamic situations of varying fairness thresholds and channel conditions. Thirdly, the effectiveness of the proposed algorithm is validated with at least 5.01% throughput improvement by comparing with proportional scheme, average scheme and Q-learning scheme.
With the emergence of innovative applications in vertical industries such as smart home and industrial automation, machine communication has shown a spurt of development. Different from the traditional human-oriented cellular communication, machine communication is characterized by strong uncertainty and abruptness, large-scale concurrent device connection as well as uneven and unsaturated data traffic. This article investigates the dynamic multiple-devices multiple-channels access for unsaturated traffic with retransmission mechanism, which is aimed at reducing the long-term data packet loss resulting from buffer overflows and transmission failure. The instant channel selection will lead to a non-negligible impact on the future decision, motivating us to model this problem as a Markov decision process. Limited by the unknown environment knowledge, we proposed a dynamic access policy based on deep reinforcement learning algorithm to optimally select the channel for transmission or keep silent for Internet-of-Things devices. Simulation results confirm that our proposed channel access strategy can reduce the collision and the packet loss of network. Furthermore, it can also work well when coexisting with the devices that adopt time division multiple access (TDMA) or ALOHA protocol.
13 February 2025 in IEEE Internet of Things Journal
Digital semantic communication has attracted considerable attention attributed to its potential for integration with modern digital communication systems, which has demonstrated significant performance gains. However, despite its ability to save transmission bandwidth, digital semantic communication can degrade the performance of tasks at the receiver, particularly in low-bitrate scenarios. In this article, we propose a novel low-bitrate digital semantic communication method based on a generative model for speech transmission to achieve high-quality reconstructed speech at low-bitrate transmission. In particular, we first investigate a multiscale semantic codec based on residual vector quantization with a generative adversary network (RVQGAN) model for extracting semantic information and obtaining high speech reconstruction quality while transmitting at a low bitrate. We then, design a channel noise suppression (CNS) module based on U-Net to alleviate the channel effect at low signal-to-noise ratio (SNR) by restoring high-quality semantic features, which is capable of improving the performance of the proposed method under challenging channel conditions. Moreover, a Transformer-based code predictor is utilized to further improve the robustness of the proposed method by accounting for both the channel impact and reconstruction quality. Finally, a three-stage training strategy is also presented in this article to ensure the effective operation of the proposed multiscale semantic codec, CNS module, and code predictor module. Experimental results demonstrate that the proposed method operating at 3 kb/s can save at least 50% of bandwidth while achieving higher speech restoration quality than the baseline method.