In recent years, the amalgamation of satellite communications and aerial platforms into space-air-ground integrated network (SAGINs) has emerged as an indispensable area of research for future communications due to the global coverage capacity of low Earth orbit (LEO) satellites and the flexible Deployment of aerial platforms. This paper presents a deep reinforcement learning (DRL)-based approach for the joint optimization of offloading and resource allocation in hybrid cloud and multi-access edge computing (MEC) scenarios within SAGINs. The proposed system considers the presence of multiple satellites, clouds and unmanned aerial vehicles (UAVs). The multiple tasks from ground users are modeled as directed acyclic graphs (DAGs). With the goal of reducing energy consumption and latency in MEC, we propose a novel multi-agent algorithm based on DRL that optimizes both the offloading strategy and the allocation of resources in the MEC infrastructure within SAGIN. A hybrid action algorithm is utilized to address the challenge of hybrid continuous and discrete action space in the proposed problems, and a decision-assisted DRL method is adopted to reduce the impact of unavailable actions in the training process of DRL. Through extensive simulations, the results demonstrate the efficacy of the proposed learning-based scheme, the proposed approach consistently outperforms benchmark schemes, highlighting its superior performance and potential for practical applications.

This paper proposes a multi-agent deep reinforcement learning-based buffer-aided relay selection scheme for an intelligent reflecting surface (IRS)-assisted secure cooperative network in the presence of an eavesdropper. We consider a practical phase model where both phase shift and reflection amplitude are discrete variables to vary the reflection coefficients of the IRS. Furthermore, we introduce the buffer-aided relay to enhance the secrecy performance, but the use of the buffer leads to the cost of delay. Thus, we aim to maximize either the average secrecy rate with a delay constraint or the throughput with both delay and secrecy constraints, by jointly optimizing the buffer-aided relay selection and the IRS reflection coefficients. To obtain the solution of these two optimization problems, we divide each of the problems into two sub-tasks and then develop a distributed multi-agent reinforcement learning scheme for the two cooperative sub-tasks, each relay node represents an agent in the distributed learning. We apply the distributed reinforcement learning scheme to optimize the IRS reflection coefficients, and then utilize an agent on the source to learn the optimal relay selection based on the optimal IRS reflection coefficients in each iteration. Simulation results show that the proposed learning-based scheme uses an iterative approach to learn from the environment for approximating an optimal solution via the exploration of multiple agents, which outperforms the benchmark schemes.

This letter proposes a deep reinforcement learning (DRL) based relay selection scheme for cooperative networks with the intelligent reflecting surface (IRS). We consider a practical phase-dependent amplitude model in which the IRS reflection amplitudes vary with the discrete phase-shifts. Furthermore, we apply the relay selection to reduce the signal loss over distance in IRS-assisted networks. To solve the complicated problem of joint relay selection and IRS reflection coefficient optimization, we introduce DRL to learn from the environment to obtain the solution and reduce the computational complexity. Simulation results show that the throughput is significantly improved with the proposed DRL-based algorithm compared to random relay selection and random reflection coefficients methods.

This paper proposes learning-based joint optimization of unmanned aerial vehicle (UAV) trajectory and reconfigurable intelligent surface (RIS) reflection coefficients in UAV-RIS-assisted cognitive non-terrestrial networks (NTNs) to enhance the secrecy performance. The practical RIS phase shift model, outdated channel state information (CSI) and interference from neighboring satellites are considered. We introduce a deep reinforcement learning (DRL) algorithm to solve the UAV trajectory optimization problem to enhance the gain from RIS. Furthermore, we propose a double cascade correlation network (DCCN) to adjust the RIS reflection coefficients in UAV trajectory optimization. Simulation results show that the proposed algorithms significantly improve the secrecy performance in UAV-RIS-assisted cognitive NTNs.

The space-air-ground integrated network (SAGIN) has become a crucial research direction in future wireless communications due to its ubiquitous coverage, rapid and flexible deployment, and multi-layer cooperation capabilities. However, integrating hierarchical federated learning (HFL) with edge computing and SAGINs remains a complex open issue to be resolved. This paper proposes a novel framework for applying HFL in SAGINs, utilizing aerial platforms and low Earth orbit (LEO) satellites as edge servers and cloud servers, respectively, to provide multi-layer aggregation capabilities for HFL. The proposed system also considers the presence of inter-satellite links (ISLs), enabling satellites to exchange federated learning models with each other. Furthermore, we consider multiple different computational tasks that need to be completed within a limited satellite service time. To maximize the convergence performance of all tasks while ensuring fairness, we propose the use of the distributional soft-actor-critic (DSAC) algorithm to optimize resource allocation in the SAGIN and aggregation weights in HFL. Moreover, we address the efficiency issue of hybrid action spaces in deep reinforcement learning (DRL) through a decoupling and recoupling approach, and design a new dynamic adjusting reward function to ensure fairness among multiple tasks in federated learning. Simulation results demonstrate the superiority of our proposed algorithm, consistently outperforming baseline approaches and offering a promising solution for addressing highly complex optimization problems in SAGINs.

This paper applies the reinforcement learning in the joint relay selection and power allocation in the secure cognitive radio (CR) relay network, where the data buffers and full-duplex jamming are applied at the relay nodes. Two cases are considered: maximizing the throughput with the delay and secrecy constraints, and maximizing the secrecy rate with the delay constraint, respectively. In both cases, the optimization relies on the buffer states, the interference to/from the primary user, and the constraints on the delay and/or secrecy. This makes it mathematically intractable to apply the traditional optimization methods. In this paper, the double deep Q-network (DDQN) is used to solve the above two optimization problems. We also apply the a-priori information in the CR network to improve the DDQN learning convergence. Simulation results show that the proposed scheme outperforms the traditional algorithm significantly.