The Internet of things (IoT), which integrates a variety of devices into networks to provide advanced and intelligent services, has to protect user privacy and address attacks such as spoofing attacks, denial of service (DoS) attacks, jamming, and eavesdropping. We investigate the attack model for IoT systems and review the IoT security solutions based on machine-learning (ML) techniques including supervised learning, unsupervised learning, and reinforcement learning (RL). ML-based IoT authentication, access control, secure offloading, and malware detection schemes to protect data privacy are the focus of this article. We also discuss the challenges that need to be addressed to implement these ML-based security schemes in practical IoT systems.

In this paper, we study an intelligent reflecting surface (IRS)-aided wireless secure communication system, where an IRS is deployed to adjust its reflecting elements to secure the communication of multiple legitimate users in the presence of multiple eavesdroppers. Aiming to improve the system secrecy rate, a design problem for jointly optimizing the base station (BS)’s beamforming and the IRS’s reflecting beamforming is formulated considering different quality of service (QoS) requirements and time-varying channel conditions. As the system is highly dynamic and complex, and it is challenging to address the non-convex optimization problem, a novel deep reinforcement learning (DRL)-based secure beamforming approach is firstly proposed to achieve the optimal beamforming policy against eavesdroppers in dynamic environments. Furthermore, post-decision state (PDS) and prioritized experience replay (PER) schemes are utilized to enhance the learning efficiency and secrecy performance. Specifically, a modified PDS scheme is presented to trace the channel dynamic and adjust the beamforming policy against channel uncertainty accordingly. Simulation results demonstrate that the proposed deep PDS-PER learning based secure beamforming approach can significantly improve the system secrecy rate and QoS satisfaction probability in IRS-aided secure communication systems.

Internet of Things (IoT) devices can apply mobile edge computing (MEC) and energy harvesting (EH) to provide high-level experiences for computational intensive applications and concurrently to prolong the lifetime of the battery. In this paper, we propose a reinforcement learning (RL) based offloading scheme for an IoT device with EH to select the edge device and the offloading rate according to the current battery level, the previous radio transmission rate to each edge device, and the predicted amount of the harvested energy. This scheme enables the IoT device to optimize the offloading policy without knowledge of the MEC model, the energy consumption model, and the computation latency model. Further, we present a deep RL-based offloading scheme to further accelerate the learning speed. Their performance bounds in terms of the energy consumption, computation latency, and utility are provided for three typical offloading scenarios and verified via simulations for an IoT device that uses wireless power transfer for energy harvesting. Simulation results show that the proposed RL-based offloading scheme reduces the energy consumption, computation latency, and task drop rate, and thus increases the utility of the IoT device in the dynamic MEC in comparison with the benchmark offloading schemes.

Unmanned aerial vehicles (UAVs) are capable of serving as flying base stations (BSs) for supporting data collection, machine learning (ML) model training, and wireless communications. However, due to the privacy concerns of devices and limited computation or communication resource of UAVs, it is impractical to send raw data of devices to UAV servers for model training. Moreover, due to the dynamic channel condition and heterogeneous computing capacity of devices in UAV-enabled networks, the reliability and efficiency of data sharing require to be further improved. In this paper, we develop an asynchronous federated learning (AFL) framework for multi-UAV-enabled networks, which can provide asynchronous distributed computing by enabling model training locally without transmitting raw sensitive data to UAV servers. The device selection strategy is also introduced into the AFL framework to keep the low-quality devices from affecting the learning efficiency and accuracy. Moreover, we propose an asynchronous advantage actor-critic (A3C) based joint device selection, UAVs placement, and resource management algorithm to enhance the federated convergence speed and accuracy. Simulation results demonstrate that our proposed framework and algorithm achieve higher learning accuracy and faster federated execution time compared to other existing solutions.

Frequency hopping-based anti-jamming techniques are not always applicable in vehicular ad-hoc networks (VANETs) due to the high mobility of onboard units (OBUs) and the large-scale network topology. In this paper, we use unmanned aerial vehicles (UAVs) to relay the message of an OBU and improve the communication performance of VANETs against smart jammers that observe the ongoing OBU and UAV communication status and even induce the UAV to use a specific relay strategy and then attack it accordingly. More specifically, the UAV relays the OBU message to another roadside unit (RSU) with a better radio transmission condition if the serving RSU is heavily jammed or interfered. The interactions between a UAV and a smart jammer are formulated as an anti-jamming UAV relay game, in which the UAV decides whether or not to relay the OBU message to another RSU, and the jammer observes the UAV and VANET strategy and chooses the jamming power accordingly. The Nash equilibria of the UAV relay game are derived to reveal how the optimal UAV relay strategy depends on the transmit cost and the UAV channel model. A hotbooting policy hill climbing (PHC)-based UAV relay strategy is proposed to help VANET resist jamming in the dynamic game without being aware of the VANET model and the jamming model. Simulation results show that the proposed relay strategy can efficiently reduce the bit error rate of the OBU message and thus increase the utility of the VANET compared with a Q-learning based scheme.

Non-orthogonal multiple access (NOMA) systems are vulnerable to jamming attacks, especially smart jammers who apply programmable and smart radio devices such as software defined radios to flexibly control their jamming strategy according to the ongoing NOMA transmission and radio environment. In this paper, the power allocation of a base station in a NOMA system equipped with multiple antennas contending with a smart jammer is formulated as a zero-sum game, in which the base station as the leader first chooses the transmit power on multiple antennas while a jammer as the follower selects the jamming power to interrupt the transmission of the users. A Stackelberg equilibrium of the anti-jamming NOMA transmission game is derived and the conditions assuring its existence are provided to disclose the impact of multiple antennas and radio channel states. A reinforcement learning based power control scheme is proposed for the downlink NOMA transmission without being aware of the jamming and radio channel parameters. The Dyna architecture that formulates a learned world model from the real anti-jamming transmission experience and the hotbooting technique that exploits experiences in similar scenarios to initialize the quality values are used to accelerate the learning speed of the Q-learning based power allocation and thus improve the communication efficiency of NOMA transmission in the presence of smart jammers. Simulation results show that the proposed scheme can significantly increase the sum data rates of users and thus the utilities compared with the standard Q-learning based strategy.