In order to mitigate the impact of explosively increasing data traffic on content request services in the Internet of Vehicles (IoV), edge caching technology is implemented in IoV to accelerate the response process of content requests and release the backhaul burden of the base station. However, the content popularity obtained by the traditional content popularity method cannot capture the requests of vehicles accurately due to the time-varying characteristics of the content popularity, which results in a relatively low cache hit ratio. Thus, this article proposes a cooperative caching strategy with content request prediction (CCCRP) in IoV, which precaches the contents requested by vehicles with greater probability in other vehicles or the roadside unit (RSU) to reduce the content acquisition delay. Specifically, vehicles are first clustered using the K-means method to simplify the process of vehicle requesting and content transmission. Then, content requests from vehicles are predicted using the long short-term memory (LSTM) networks according to the historical content request information. Finally, reinforcement learning method is adopted to solve the objective function to obtain the optimal caching decision, which improves the Quality of Service (QoS) of vehicle requests. Simulation results demonstrate that CCCRP can improve the cache hit ratio and reduce content acquisition delay effectively. For example, the cache hit ratio of CCCRP can be increased by 5% and 7% compared to the traditional LFU and LRU caching strategies when the Zipf parameter equals 0.7, respectively.

To support diverse Internet of vehicles (IoV) services with different quality of service (QoS) requirements, network slicing is applied in vehicular networks to establish multiple logically isolated networks on common physical network infrastructure. However, dynamic and efficient radio access network (RAN) slicing adapting to the dynamics of vehicular networks remains challenging. The diverse applications make multi-dimensional resource requirements, which will result in the resource allocation more complicated. In addition, the system needs to frequently adjust the resources of slices, which will cause additional slicing overhead. Thus, to solve the above problems, we propose a resource allocation strategy by using multi-agent reinforcement learning to allocate resources in vehicular networks. Firstly, the cost composition of RAN slicing is analyzed, and the optimization problem is formulated to minimize the long-term system cost. Then, we transform the resource allocation problem into a partially observable Markov decision process. Finally, we propose a multi-agent deep deterministic policy gradient based resource allocation algorithm to solve it. All base stations are treated as independent agents, and they cooperatively allocate spectrum and computing resources. Simulation results show that the proposed strategy reduces the system cost effectively compared to the benchmarks, and the average QoS satisfaction rate achieves 96.5%.

Multi-access mobile edge computing (MEC) is envisioned as a key enabling technology to support compute-intensive and delay-sensitive applications in railway Internet of Things (RIoT) networks. However, the time-varying channel variations in RIoT scenarios make it challenging to achieve efficient resource allocation. The emerging deep reinforcement learning (DRL) is able to respond to the above-mentioned challenge. In this paper, with the aim of reducing the total computational cost (weighted sum of consumed energy and delay), we investigate the dynamic resource management issue of joint subcarrier assignment, offloading ratio, power allocation and computation resource allocation in multi-access MEC assisted RIoT networks. To address this intractable mixed integer nonlinear programming issue, we put forward a hybrid DRL (HDRL) scheme, which is an integration of deep double Q-learning (DDQN) and deep deterministic policy gradient (DDPG). The HDRL algorithm is capable of learning the advisable strategies for actions including discrete-continuous hybrid variables. In HDRL algorithm, DDQN plays the role of making subcarrier assignment decision, and DDPG plays the role of making offloading ratio, power allocation as well as computation resource allocation decisions. Numerical results demonstrate that HDRL scheme can yield much less computational cost than the existing baselines for multi-access MEC assisted RIoT networks. In addition, the HDRL scheme is close to the near-optimal performance with comparatively low execution time.

The latest research of network and computing contributes greatly to the development of vehicular networks. However, in existing works, these two important enabling technologies are studied separately. To reduce the delay, in this paper, we propose a multi-platform intelligent offloading and resource allocation algorithm which can dynamically organize the computing resources to improve the performance of the next-generation vehicular networks. Considering the task calculation problem, the K-nearest neighbor algorithm is used to select the task offloading platform (i.e., cloud computing, mobile edge computing, or local computing). For the computational resource allocation problem and system complexity in non-local computing, reinforcement learning is used to solve the optimization problem of resource allocation. The simulation results show that compared with the baseline algorithm that all tasks are offloaded to the local or mobile edge computing server, the resource allocation scheme achieves a significant reduction in latency cost, and the average system cost can be saved by 80%.

To support the diversified Quality of Service (QoS) requirements of application scenarios, network slicing has been introduced in the mobile cellular network. It allows mobile cellular network operators to accomplish the creation of multiple logically isolated networks on common network infrastructure flexibly depending on specified demands. Meanwhile, in Internet of Vehicles (IoV), it is very intractable to supply a stable QoS for the vehicles, especially for the dynamic vehicular environments. Thus, we investigate the IoV slicing problem in this article, and propose a QoS guaranteed network slicing orchestration, namely, the long short-term memory-based deep deterministic policy gradient algorithm (LSTM-DDPG), to ensure the stable performance for the slices. Specifically, we first decouple the resource allocation problem into two subproblems. After that, the deep learning and reinforcement learning (RL) are used to allocate resources collaboratively to solve these two questions. We use deep learning LSTM to track the characteristic of the long-term vehicular environment changing, and the RL algorithm DDPG is utilized for online resource tuning. Extensive simulations have proved the effectiveness of the LSTM-DDPG, which can offer stable QoS to the vehicles with a probability greater than 92%. We also demonstrated the adaptiveness of the proposed orchestration with different slicing environments, and the performance is always optimal compared to that of other algorithms.

Using higher frequency bands (e.g., millimeter waves) to provide higher data rate is an effective way to eliminate performance bottleneck for future wireless networks, particularly for cellular networks based high speed railway (HSR) wireless communication systems. However, higher frequency bands suffer from significant path loss and narrow-beam coverage, which pose serious challenges in cellular networks, especially under HSR scenario. Meanwhile, as one of the key performance indexes (KPIs) of ultra-reliable and low-latency communications (URLLC) in 5G systems, network reliability should be guaranteed to provide steady reliable data transmission along the railway, especially when safety-critical railway signaling information is delivered. In this paper, we propose a novel beamforming scheme, namely, optimal non-uniform steady mmWave beamforming, to guarantee the network reliability under an interleaved redundant coverage architecture for future HSR wireless systems. Moreover, we develop a bisection based beam boundary determination (BBBD) method to determine the service area of each predefined RF beam. Finally, we demonstrate that the proposed optimal non-uniform steady mmWave beamforming can provide steady reliable data transmissions along the railway, and the network reliability requirements can be guaranteed when the proposed BBBD method is used. We expect that our optimal non-uniform steady mmWave beamforming provides a promising solution for future HSR wireless systems.