Ultra-dense network (UDN) is considered as one of the key technologies for the explosive growth of mobile traffic demand on the Internet of Things (IoT). It enhances network capacity by deploying small base stations in large quantities, but it simultaneously causes great energy consumption. In this paper, we use energy harvesting (EH) and energy cooperation technologies to maximize system throughput and save energy. Considering that the energy arrival process and channel information are not available a-priori, we propose an optimal deep reinforcement learning (DRL) algorithm to solve this average throughput maximization problem over finite horizon. We also propose a multi-agent deep reinforcement learning method to solve the dimensionality problem caused by the expansion of the state and action dimension. Finally, we compare these algorithms with two traditional algorithms, greedy algorithm and conservative algorithm. The numerical results show that the proposed algorithms are valid and effective in increasing system average throughput on long term.
With the continuous advancement of the networking process of the IoV (Internet of Vehicles), cooperative driving is gradually becoming possible. However, the unstable characteristics of V2X (vehicle-to-everything) communication cannot be ignored in the research of cooperative driving control strategies. To provide data support to the communication-related compensation strategy, the characteristics of communication delays under three different scenarios are presented in this paper. The authors describe the test system and procedures and the outcomes of the testing. The findings in this study shed light on metrics of packet loss, burst packet loss, and reliability.