Satellite-terrestrial spectrum sharing is an effective method to alleviate the scarcity of spectrum resources. However, with the dense deployment of terrestrial systems, the satellite downlink beam covers multiple terrestrial base stations (BSs) and interacts with terrestrial downlinks during spectrum sharing. The spectrum sharing users encounter co-channel interference of inter-system, inter-cell and intra-cell, posing significant challenges for downlink spectrum sharing in converged system. In this paper, we propose a cooperative multichannel spectrum sharing framework where the multiple terrestrial BSs downlinks share multichannel spectrum resources of low earth orbit (LEO) satellite while guaranteeing the rate requirements of all shared users and aiming at improving spectrum efficiency of converged system. We formulate the cooperative satellite-terrestrial spectrum sharing strategy as a global spectrum efficiency maximization problem for the converged system, achieved through multichannel assignment and power allocation optimization for terrestrial shared users. To solve the non-convex optimization problem, we provide a dynamic and collaborative spectrum sharing (DCSS) algorithm based on multi-agent deep reinforcement learning (MADRL), which includes DDQN-based multichannel assignment and DDPG-based power allocation. DCSS transforms the problem to a decentralized partially observable Markov decision processes (Dec-POMDP) and adopts centralized-training-distributed-execution (CTDE) framework to address the challenge of acquiring global information in real-time. Specifically, each terrestrial BS acts as an agent with a globally trained two-layer joint deep neural network model that can adaptively allocate the multichannel and power based on observed system local information. Extensive simulation results demonstrate that DCSS algorithm outperforms the existing typical algorithms.

With the rapid development of the mobile Internet and the Internet of Things, the number of mobile communication services has grown rapidly. When multiple different types of networks cover the same region, it is important to decide which one users connect to, known as the network selection problem. In this paper, we explore the optimal network selection problem in 5G ultra-dense network. We consider several different types of transmission data such as session, media, background and interactive, which conclude different QoS requirements. And then, we formulate the network selection problem as an MDP model in ultra-dense system, and propose NS-MDP algorithm which aims to obtain best target network by calculating the benefits of the utility function. NS-MDP algorithm takes into account user data requirements, current system status, and network load conditions. Comparison experiments with Best-Rate, Random and Greedy_AHP strategies, show that NS-MDP algorithm's average throughput is 8.6%, 17.8% and 20.5% higher than them, and NS-MDP can reduce blocking rate by 33.3%, 17.6%, and 37.7%.

Spectrum sharing-enabled multi-beam management is an effective method to enhance global coverage and spectrum efficiency of satellite-terrestrial integrated network (STIN). However, uneven timevarying traffic, high dynamic network topology, and multi-domain interference pose significant challenges for dynamic beam patterning in multi-LEO STINs. In this paper, we propose a spectrum sharing-enabled cooperative dynamic beam pattern framework where multi-beams of multiple LEO satellites can either fully reuse satellite frequency or share remote terrestrial spectrum. We formulate the cooperative dynamic beam pattern strategy as a joint utility maximization problem of throughput, beam queue load, and delay fairness, achieved through beam assignment, sharing model and power allocation optimization for multi-satellites. To solve the non-convex optimization problem, we provide a collaborative and dynamic beam pattern (CDBP) algorithm based on multi-agent deep reinforcement learning (MADRL). Specifically, CDBP transforms the problem to a decentralized partially observable Markov decision processes (Dec-POMDP) and adopts centralized-training-distributed-execution (CTDE) to address challenge of acquiring global information in multi-satellites. Then, long short-term memory (LSTM) network is introduced into DRL of proximal policy optimization (PPO) model, named MA-LSTM-PPO, which captures temporal dependencies of trajectory and beam queue, guiding to drive optimal beam pattern. Extensive simulation results demonstrate that CDBP algorithm can effectively reduce average queuing delay by 5-15ms compared with existing typical schemes, and adapt to uneven traffic demands in STINs.