In this paper, we investigate resource allocation with edge computing in Internet of Things (IoT) networks via machine learning approaches. Edge computing is playing promising role in IoT networks by providing computing capabilities close to users. However, the massive number of users in IoT networks requires sufficient spectrum resource to transmit their computation tasks to a edge server, while IoT users were developed to have more powerful computation ability recently, which makes it possible for them to execute some tasks locally. Then the design of computation task offloading policies for such IoT edge computing systems remains challenging. In this paper, centralized user clustering is explored to group the IoT users into different clusters according to users’ priorities. The cluster with the highest priority is assigned to offload computation tasks and executed at the edge server, while the lowest priority cluster executes computation tasks locally. For the other clusters, the design of distributed task offloading policies for IoT users is modelled by a Markov decision process, where each IoT user is considered as an agent which makes a series of decisions on task offloading by minimizing the system cost based on the environment dynamics. To deal with the curse of high dimensionality, we use deep Q network to learn the optimal policy in which deep neural network is used to approximate the Q-function in Q-learning. Simulations show that users are grouped into clusters with optimal number of clusters. Moreover, our proposed computation offloading algorithm outperforms the other baseline schemes under same system costs.

Cellular-connected unmanned aerial vehicle (UAV) with flexible deployment is foreseen to be a major part of the sixth generation (6G) networks. The UAVs connected to the base station (BS), as aerial users (UEs), could exploit machine learning (ML) algorithms to provide a wide range of advanced applications, like object detection and video tracking. Conventionally, the ML model training is performed at the BS, known as centralized learning (CL), which causes high communication overhead due to the transmission of large datasets, and potential concerns about UE privacy. To address this, distributed learning algorithms, including federated learning (FL) and split learning (SL), were proposed to train the ML models in a distributed manner via only sharing model parameters. FL requires higher computational resource on the UE side than SL, while SL has larger communication overhead when the local dataset is large. To effectively train an ML model considering the diversity of UEs with different computational capabilities and channel conditions, we first propose a novel distributed learning architecture, a hybrid split and federated learning (HSFL) algorithm by reaping the parallel model training mechanism of FL and the model splitting structure of SL. We then provide its convergence analysis under non-independent and identically distributed (non-IID) data with random UE selection scheme. By conducting experiments on training two ML models, Net and AlexNet, in wireless UAV networks, our results demonstrate that the HSFL algorithm achieves higher learning accuracy than FL and less communication overhead than SL under IID and non-IID data, and the learning accuracy of HSFL algorithm increases with the increasing number of the split training UEs. We further propose a Multi-Arm Bandit (MAB) based best channel (BC) and best 2-norm (BN2) (MAB-BC-BN2) UE selection scheme to select the UEs with better wireless channel quality and larger local model updates for model training in each round. Numerical results demonstrate it achieves higher learning accuracy than BC, MAB-BC and MAB-BN2 UE selection scheme under non-IID, Dirichlet-nonIID and Dirichlet-Imbalanced data.

The space-air-ground integrated network (SAGIN) aims to provide seamless wide-area connections, high throughput and strong resilience for 5G and beyond communications. Acting as a crucial link segment of the SAGIN, unmanned aerial vehicle (UAV)-satellite communication has drawn much attention. However, it is a key challenge to track dynamic channel information due to the low earth orbit (LEO) satellite orbiting and three-dimensional (3D) UAV trajectory. In this paper, we explore the 3D channel tracking for a Ka-band UAV-satellite communication system. We firstly propose a statistical dynamic channel model called 3D two-dimensional Markov model (3D-2D-MM) for the UAV-satellite communication system by exploiting the probabilistic insight relationship of both hidden value vector and joint hidden support vector. Specifically, for the joint hidden support vector, we consider a more realistic 3D support vector in both azimuth and elevation direction. Moreover, the spatial sparsity structure and the time-varying probabilistic relationship between degree patterns named the spatial and temporal correlation, respectively, are studied for each direction. Furthermore, we derive a novel 3D dynamic turbo approximate message passing (3D-DTAMP) algorithm to recursively track the dynamic channel with the 3D-2D-MM priors. Numerical results show that our proposed algorithm achieves superior channel tracking performance to the state-of-the-art algorithms with lower pilot overhead and comparable complexity.

In this paper, efficient resource allocation for the uplink transmission of wireless powered IoT networks is investigated. We adopt LoRa technology as an example in the IoT network, but this work is still suitable for other communication technologies. Allocating limited resources, like spectrum and energy resources, among a massive number of users faces critical challenges. We consider grouping wireless powered IoT users into available channels first and then investigate power allocation for users grouped in the same channel to improve the network throughput. Specifically, the user grouping problem is formulated as a many to one matching game. It is achieved by considering IoT users and channels as selfish players which belong to two disjoint sets. Both selfish players focus on maximizing their own utilities. Then we propose an efficient channel allocation algorithm (ECAA) with low complexity for user grouping. Additionally, a Markov Decision Process (MDP) is used to model unpredictable energy arrival and channel conditions uncertainty at each user, and a power allocation algorithm is proposed to maximize the accumulative network throughput over a finite-horizon of time slots. By doing so, we can distribute the channel access and dynamic power allocation local to IoT users. Numerical results demonstrate that our proposed ECAA algorithm achieves near-optimal performance and is superior to random channel assignment, but has much lower computational complexity. Moreover, simulations show that the distributed power allocation policy for each user is obtained with better performance than a centralized offline scheme.

To boost the performance of wireless communication networks, unmanned aerial vehicles (UAVs) aided communications have drawn dramatically attention due to their flexibility in establishing the line of sight (LoS) communications. However, with the blockage in the complex urban environment, and due to the movement of UAVs and mobile users, the directional paths can be occasionally blocked by trees and high-rise buildings. Intelligent reflection surfaces (IRSs) that can reflect signals to generate virtual LoS paths are capable of providing stable communications and serving wider coverage. This is the first paper that exploits a three-dimensional geometry dynamic channel model in IRS- assisted UAV-enabled communication system. Moreover, we develop a novel deep learning based channel tracking algorithm consisting of two modules: channel pre-estimation and channel tracking. A deep neural network with off-line training is designed for denoising in the pre-estimation module. Moreover, for channel tracking, a stacked bi-directional long short term memory (Stacked Bi-LSTM) is developed based on a framework that can trace back historical time sequence together with bidirectional structure over multiple stacked layers. Simulations have shown that the proposed channel tracking algorithm requires fewer epochs to convergence compared to benchmark algorithms. It also demonstrates that the proposed algorithm is superior to different benchmarks with small pilot overheads and comparable computation complexity.

By employing powerful edge servers for data processing, mobile edge computing (MEC) has been recognized as a promising technology to support emerging computation-intensive applications. Besides, non-orthogonal multiple access (NOMA)-aided MEC system can further enhance the spectral-efficiency with massive tasks offloading. However, with more dynamic devices brought online and the uncontrollable stochastic channel environment, it is even desirable to deploy appealing technique, i.e., intelligent reflecting surfaces (IRS), in the MEC system to flexibly tune the communication environment and improve the system energy efficiency. In this paper, we investigate the joint offloading, communication and computation resource allocation for the IRS-assisted NOMA MEC system. We first formulate a mixed integer energy efficiency maximization problem with system queue stability constraint. We then propose the Lyapunov-function-based Mixed Integer Deep Deterministic Policy Gradient (LMIDDPG) algorithm which is based on the centralized reinforcement learning (RL) framework. To be specific, we design the mixed integer action space mapping which contains both continuous mapping and integer mapping. Moreover, the award function is defined as the upper-bound of the Lyapunov drift-plus-penalty function. To enable end devices (EDs) to choose actions independently at the execution stage, we further propose the Heterogeneous Multi-agent LMIDDPG (HMA-LMIDDPG) algorithm based on distributed RL framework with homogeneous EDs and heterogeneous base station (BS) as heterogeneous multi-agent. Numerical results show that our proposed algorithms can achieve superior energy efficiency performance to the benchmark algorithms while maintaining the queue stability. Specially, the distributed structure HMA-LMIDDPG can acquire more energy efficiency gain than the centralized structure LMIDDPG.