The next-generation wireless network is expected to use low- earth orbit (LEO) satellite networks to deliver seamless and high-capacity global communications services. Due to the high-speed mobility of LEO satellites, massive and frequent handovers inevitably occur. Moreover, handover becomes more complicated with the ever-growing constellation scale, number of mobile terminals (MTs), and demands for emerging delay-sensitive applications. In this paper, a decentralized Markov decision process (DEC-MDP) is adopted to formulate the handover problem in the LEO satellite network with finite bursty traffic. The target is maximizing the total reward associated with the service revenue and the cost of handover and packet loss. To deal with the high computational complexity caused by the large state space and action space, the solution is designed using a multi-agent double deep Q-network (MADDQN) with fully decentralized framework, which also allows each MT to train and use an individual local DDQN to avoid load imbalance between satellites. Further, to alleviate the non-stationary issue of the environment in parallel learning, multi-agent fingerprints are applied in MADDQN, and the proposed algorithm is called multi-agent fingerprints-enhanced double deep Q-network-based distributed intelligent handover (MAF-DDQN-DIH) mechanism. The implementation of MAF-DDQN-DIH in practical communication systems are discussed, and the corresponding communication overhead and computational complexity are analyzed. Simulation results demonstrate that the designed multi-agent fingerprints are effective and the proposed MAF-DDQN-DIH algorithm outperforms the comparison handover algorithms in terms of the total reward.

With the rapid development of 5G technology in recent years, multimedia communication services, such as live online and short video, have received wide attention and become the important means of people’s daily social intercourse. However, the rapid growth of multimedia communication demands pose greater challenges to both the wireless network communication capacity and the network processing capacity. Mobile Edge Computing (MEC) is widely regarded as a promising technology to cope with the above challenges. To satisfy the growing demands and improve the quality of experience for users, it is in urge need to seek the effective and efficient task assignment and computing offloading strategy for MEC networks. In this paper, we focused on the multimedia services which need to be processed, uploaded and shared in the network and research the long-term task assignment and resource coordination problem. We formulate the optimization problem as a stochastic optimization problem with the aim of the minimizing the time-average energy consumption of the system. By using the Lyapunov optimization technique, we decompose the original problem into several subproblems which can be solved with current system information and low computational complexities. On this basis, we propose an online energy-efficient task assignment and computing offloading strategy to adaptively decide the task assignment, coordinate and optimize the wireless and computation resource allocation by taking the dynamic wireless condition and service delay constraints into consideration. Extensive simulation results show that our proposed algorithm can achieve considerate energy consumption and delay performances under different conditions.

In multi-beam low-earth orbit (LEO) satellite networks, frequent handovers between intra-satellite and inter-satellite beams are inevitable. In this letter, we design a beam handover strategy based on the multi-objective reinforcement learning (MORL) method to achieve seamless and effective handover between multiple beams of LEO satellites. We first model the handover optimization problem of the multi-beam LEO satellite networks as a multi-objective optimization (MOO) problem to jointly maximize throughput, minimize the handover frequency, and keep the network load balanced. On this basis, we convert the MOO problem into a multi-objective Markov decision process (MOMDP), and utilize an MORL method, called multi-objective deep Q-learning network (MODQN), to learn and achieve the optimal solution. Simulation results show the effectiveness and superiority of the proposed handover scheme.

In the Internet of Vehicles (IoV) with network slicing functions, both the inter-slice resource allocation and the intra-slice task scheduling have received extensive attention. However, the joint optimization of these two problems has not been fully studied. This letter focuses on the complexity issue of the joint resource allocation and task scheduling problem in the IoV with both ultra-reliable low-latency communication (URLLC) and enhanced mobile broadband (eMBB) slices. A heterogeneous Markov decision process (HMDP) which models the resource allocation process and the task scheduling process in two different layers is proposed. Benefiting from the two-layer structure of the HMDP, the size of action space is significantly reduced, and the asynchronous decisions of different sub-MDPs are enabled. The corresponding layered deep reinforcement learning (DRL) architecture is also designed to solve the HMDP-based optimization problem. Simulation results show that the layered DRL-based algorithm outperforms other methods.

With the rapid development of mobile communication technology, short video applications, which combine the features of both social and multimedia applications, have become more and more popular. However, the transmission of short videos poses great challenges to the existing mobile networks. In this paper, mobile edge computing is adopted to provide content caching of short videos close to end users. To improve the quality of service, we take both the quality level of the video and the long-term wireless network transmission performance into consideration. The joint video quality selection and radio bearer control optimization problem is formulated as a Markov decision process, aiming at maximizing the long-term video quality profit and minimizing the cost of bearers and the penalty of latency. Deep reinforcement learning is used as the solution and the policy gradient based quality selection and radio bearer control method is proposed. The REINFORCE algorithm with baseline is used to train the policy network, and the episodic simulations are built to obtain the training samples. Different weight coefficients of the objective function are configured. Training results show that the proposed method can achieve the best accumulated value among all the comparison methods. When the weight coefficients are changed, the training processes can lead the policy networks to obtain proper trade-off between different objective factors. Moreover, the performance of the trained policy network is evaluated with different short video request arriving rates. Testing results show that the proposed method performs well when the arriving rates vary in a certain range.

With the rapid increase of mobile data traffic, efficient content delivery in wireless networks is increasingly important to provide broadcast and multicast services. To reduce network cost and enrich end-user quality of experience, Internet service providers (ISPs) and content providers (CPs) are expected to collaborate to improve content delivery services. However, the influence of content popularity has been generally overlooked, and profit split problem between ISPs and CPs has not been studied. We investigate the novel economic behaviors between ISPs and CPs in the presence of edge caches, and formulate the profit split problem as a centralized model, which can achieve optimal content caching and maximal network benefits. A Stackelberg game is designed to obtain its distributed win–win solution, where a feasible backward induction is proposed and the influence of edge caches and content popularity is analyzed. Simulation results show that the performance of our proposed Stackelberg game model is comparable to the centralized alternative and much better than existing ISP-CP cooperative schemes not considering cache deployment in the network.