Mobile edge computing systems help improve the performance of computational-intensive applications on mobile devices and have to resist jamming attacks and heavy interference. In this paper, we present a reinforcement learning based mobile offloading scheme for edge computing against jamming attacks and interference, which uses safe reinforcement learning to avoid choosing the risky offloading policy that fails to meet the computational latency requirements of the tasks. This scheme enables the mobile device to choose the edge device, the transmit power and the offloading rate to improve its utility including the sharing gain, the computational latency, the energy consumption and the signal-to-interference-plus-noise ratio of the offloading signals without knowing the task generation model, the edge computing model, and the jamming/interference model. We also design a deep reinforcement learning based mobile offloading for edge computing that uses an actor network to choose the offloading policy and a critic network to update the actor network weights to improve the computational performance. We discuss the computational complexity and provide the performance bound that consists of the computational latency and the energy consumption based on the Nash equilibrium of the mobile offloading game. Simulation results show that this scheme can reduce the computational latency and save energy consumption.

With the dynamic renewable energy generation and power demand, microgrids (MGs) exchange energy with each other to reduce their dependence on power plants. In this paper, we present a reinforcement learning based MG energy trading scheme to choose the electric energy trading policy according to the predicted future renewable energy generation, the estimated future power demand, and the MG battery level. This scheme designs a deep reinforcement learning based energy trading algorithm to address the supply-demand mismatches problem for the smart grid with a large number of MGs without relying on the renewable energy generation model and the power demand model of other MGs. The performance bound of the MG utility and the dependence on the power plant is provided and verified via simulations. Simulation results based on a smart grid with three MGs using the wind speed data from Hong Kong Observation and the electricity prices from ISO New England show that this scheme significantly reduces the average power plant schedule and thus increases the MG utility in comparison with the existing trading scheme.

Sixth-generation (6G) cellular systems will have an inherent vulnerability to physical (PHY)-layer attacks and privacy leakage, due to the large-scale heterogeneous networks with booming time-sensitive applications. Important wireless techniques including non-orthogonal multiple access, mobile edge computing, millimeter-wave, massive multiple-input and multiple-output, visible light communication, terahertz, and intelligent reflecting surface can improve the spectrum efficiency and quality-of-service but will raise challenges for the 6G PHY and cross-layer security and privacy protection. Existing optimization based PHY and cross-layer security and privacy protection schemes such as the convex optimization method have to rely on accurate attack patterns and strategies and thus suffer from performance degradation in 6G systems that have shorter communication latency, more devices and higher spectrum efficiency than 5G. Reinforcement learning (RL) algorithms help wireless devices optimize their security policies to enhance the security performance in dynamic networks against smart attacks without depending on the attack model. Therefore, this article provides a comprehensive survey on the RL based 6G PHY cross-layer security and privacy protection. In this article, we investigate the potential attacks in 6G systems and discuss the PHY cross-layer security solutions. A brief overview of reinforcement learning algorithms is provided. Afterward, we review the RL based PHY-layer security and privacy protection and discuss how to apply RL algorithms in 6G security scenarios, especially focusing on the game with jammers, eavesdroppers, spoofers and inference attackers. The RL based security solutions for unmanned aerial vehicles (UAVs) and cross-layer scenarios are also reviewed. The future research directions are identified and the corresponding RL based potential solutions are discussed for 6G.

In controller area networks (CANs), electronic control units (ECUs) such as telematics ECUs and on-board diagnostic ports must protect the message exchange from spoofing attacks. In this paper, we propose a CAN bus authentication framework that exploits physical layer features of the messages, including message arrival intervals and signal voltages, and applies reinforcement learning to choose the authentication mode and parameter. By applying the Dyna architecture and using a double estimator, this scheme improves the utility in terms of authentication accuracy without changing the CAN bus protocol or the ECU components and requiring knowledge of the spoofing model. We also propose a deep learning version to further improve the authentication efficiency for the CAN bus. The learning scheme applies a hierarchical structure to reduce the exploration time, and uses two deep neural networks to compress the high-dimensional state space and to fully exploit the physical authentication experiences. We provide the computational complexity and the performance analysis. Experimental results verify the theoretical analysis and show that our proposed schemes significantly improve the authentication accuracy as compared with benchmark schemes.

Most safe reinforcement learning (RL) algorithms depend on the accurate reward that is rarely available in wireless security applications and suffer from severe performance degradation for the learning agents that have to choose the policy from a large action set. In this paper, we propose a safe RL algorithm, which uses a policy priority-based hierarchical structure to divide each policy into sub-policies with different selection priorities and thus compresses the action set. By applying inter-agent transfer learning to initialize the learning parameters, this algorithm accelerates the initial exploration of the optimal policy. Based on a security criterion that evaluates the risk value, the sub-policy distribution formulation avoids the dangerous sub-policies that cause learning failure such as severe network security problems in wireless security applications, e.g., Internet services interruption. We also propose a deep safe RL and design four deep neural networks in each sub-policy selection to further improve the learning efficiency for the learning agents that support four convolutional neural networks (CNNs): The Q-network evaluates the long-term expected reward of each sub-policy under the current state, and the E-network evaluates the long-term risk value. The target Q and E-networks update the learning parameters of the corresponding CNN to improve the policy exploration stability. As a case study, our proposed safe RL algorithms are implemented in the anti-jamming communication of unmanned aerial vehicles (UAVs) to select the frequency channel and transmit power to the ground node. Experimental results show that our proposed schemes significantly improve the UAV communication performance, save the UAV energy and increase the reward compared with the benchmark against jamming.