This paper investigates the UAV-assisted wireless powered IoT system, where a UAV takes off from a data center, flies to each of the ground sensor nodes (SNs) in order to transfer energy and collect data form the SNs, and then returns to the data center. For such a system, an optimization problem is formulated to minimize the average age of information (AoI) of the data collected from all ground SNs. Since the average AoI depends on the UAV’s trajectory, the time required for energy harvesting (EH) and data collection for each SN, these factors need to be optimized jointly. Moreover, instead of the traditional linear EH model, we employ a non-linear model because the behavior of the EH circuits are non-linear by nature. To solve this non-convex problem, we propose to decompose it into two sub-problems, i.e., a joint energy transfer and data collection time allocation problem and a UAV’s trajectory planning problem. For the first sub-problem, we prove that it is convex and give an optimal solution by using Karush-Kuhn-Tucker (KKT) conditions. This solution is used as the input for the second sub-problem, and we solve optimally it by designing dynamic programming (DP) and ant colony (AC) heuristic algorithms. Simulation results show that the DP-based algorithm obtains the minimal average AoI of the system, and the AC-based heuristic finds solutions with near-optimal average AoI. The results also reveal that the average AoI increases as the flying altitude of the UAV increases and linearly with the size of the collected data at each ground SN.

This paper investigates reconfigurable intelligent surface (RIS)-assisted simultaneous wireless information and power transfer (SWIPT) networks with rate splitting multiple access (RSMA). An energy efficiency (EE) maximization problem is formulated subject to the power budget at the transmitter and the quality of service (QoS) requirements of both information communication and energy harvesting, where the beamforming vectors, the power splitting (PS) ratios, the common message rates, and the discrete phase shifts are jointly optimized. To tackle the non-convex problem with both discrete and continuous variables, a deep reinforcement learning-based approach is proposed with the proximal policy optimization (PPO) framework. Different from traditional optimization approaches which optimizes the beamforming vectors and phase shifts separately and alternatively, our proposed PPO-based approach optimizes all the variables in unison. Besides, to perform beamforming design in action space, the beamforming vectors for the common stream and the private stream are respectively designed based on the maximum-ratio transmission and the zero forcing to enhance both energy and information transmission. To evaluate the performance of the PPO-based approach, a successive convex approximation (SCA) and Dinkelbach’s method based solution scheme (named SCA-D scheme) is also presented. Simulation results show that the system EE obtained by the proposed PPO-based approach is close to that obtained by the SCA-D scheme while outperforming various benchmarks. The RSMA contributes to the EE of the system greatly compared with traditional scheme. As for the case of time-varying channels, the proposed PPO-based approach is with much smaller running time by only sacrificing a slight EE performance compared with the SCA-D scheme.

This article investigates the unmanned-aerial-vehicle (UAV)-enabled wireless powered cooperative mobile edge computing (MEC) system, where a UAV installed with an energy transmitter (ET) and an MEC server provides both energy and computing services to sensor devices (SDs). The active SDs desire to complete their computing tasks with the assistance of the UAV and their neighboring idle SDs that have no computing task. An optimization problem is formulated to minimize the total required energy of UAV by jointly optimizing the CPU frequencies, the offloading amount, the transmit power, and the UAV’s trajectory. To tackle the nonconvex problem, a successive convex approximation (SCA)-based algorithm is designed. Since it may be with relatively high computational complexity, as an alternative, a decomposition and iteration (DAI)-based algorithm is also proposed. The simulation results show that both proposed algorithms converge within several iterations, and the DAI-based algorithm achieve the similar minimal required energy and optimized trajectory with the SCA-based one. Moreover, for a relatively large amount of data, the SCA-based algorithm should be adopted to find an optimal solution, while for a relatively small amount of data, the DAI-based algorithm is a better choice to achieve smaller computing energy consumption. It also shows that the trajectory optimization plays a dominant factor in minimizing the total required energy of the system and optimizing acceleration has a great effect on the required energy of the UAV. Additionally, by jointly optimizing the UAV’s CPU frequencies and the amount of bits offloaded to UAV, the minimal required energy for computing can be greatly reduced compared to other schemes and by leveraging the computing resources of idle SDs, the UAV’s computing energy can also be greatly reduced.

Vehicular edge computing (VEC) can learn and cache most popular contents for vehicular users (VUs) in the roadside units (RSUs) to support real-time vehicular applications. Federated learning (FL) can protect VUs' privacy by sharing vehicles' local models instead of data. In traditional FL, the global model is periodically updated by aggregating all vehicles' local models. However, vehicles may frequently been driven out of the coverage area of VEC before they finish the local model training and thus the traditional FL cannot upload all local models as expected, which would degrade the accuracy of global model. The asynchronous FL can be performed without aggregating all vehicles' local models, thus more local models can be uploaded to improve the accuracy of global model. The vehicle mobility significantly impacts the asynchronous FL. There is no published work considering the vehicle mobility to design the cooperative caching in VEC based on asynchronous FL. In addition, the caching capacity of RSU is limited and the size of the predicted popular contents usually exceeds the cache capacity of RSU. Hence, VEC should cache the predicted popular contents in different RSUs while considering content transmission delay. In this paper, we consider vehicle mobility and propose a cooperative caching scheme in the VEC based on asynchronous federated and deep reinforcement learning (CAFR). We first consider vehicle mobility and propose an asynchronous FL algorithm to obtain an accurate global model, and then propose an algorithm to predict popular contents based on the global model. In addition, we consider vehicle mobility and propose a deep reinforcement learning (DRL) algorithm to obtain the optimal cooperative caching location for the predicted popular contents in order to optimize the content transmission delay. Extensive experimental results have demonstrated that CAFR scheme outperforms other baseline caching schemes.

As an emerging technique, mobile-edge computing (MEC) introduces a new scheme for various distributed communication-computing systems, such as industrial Internet of Things (IoT), vehicular communication, smart city, etc. In this work, we mainly focus on the timeliness of the MEC systems where the freshness of the data and computation tasks is significant. First, we formulate a kind of age-sensitive MEC models and define the average Age-of-Information (AoI) minimization problems of interests. Then, a novel mixed-policy-based multimodal deep reinforcement learning (RL) framework, called heterogeneous multiagent actor–critic (H-MAAC), is proposed as a paradigm for joint collaboration in the investigated MEC systems, where edge devices and center controller learn the interactive strategies through their own observations. To improve the system performance, we develop the corresponding online algorithm by introducing the edge federated learning mode into the multiagent cooperation whose advantages on learning convergence can be guaranteed theoretically. To the best of our knowledge, it is the first joint MEC collaboration algorithm that combines the edge federated mode with the multiagent actor–critic RL. Furthermore, we evaluate the proposed approach and compare it with popular RL-based methods. As a result, the proposed algorithm not only outperforms the baselines on average system age, but also promotes the stability of training process. Besides, the simulation outcomes provide several insights for collaboration designs over MEC systems.

Heavy data load and wide cover range have always been crucial problems for big data processing in internet of things (IoT). Recently, mobile-edge computing (MEC) and unmanned aerial vehicle base stations (UAV-BSs) have emerged as promising techniques in IoT. In this paper, we propose a three-layer online data processing network based on MEC technique. On the bottom layer, raw data are generated by distributed sensors with local information. Upon them, UAV-BSs are deployed as moving MEC servers, which collect data and conduct initial steps of data processing. On top of them, a center cloud receives processed results and conducts further evaluation. For online processing requirements, the edge nodes should stabilize delay to ensure data freshness. Furthermore, limited onboard energy poses constraints to edge processing capability. In this paper, we propose an online edge processing scheduling algorithm based on Lyapunov Optimization. In cases of low data rate, it tends to reduce edge processor frequency for saving energy. In the presence of high data rate, it will smartly allocate bandwidth for edge data offloading. Meanwhile, hovering UAV-BSs bring a large and flexible service coverage, which results in path planning issue. In this paper, we also consider this problem and apply deep reinforcement learning to develop an online path planning algorithm. Taking observations of around environment as input, a CNN network is trained to predict action rewards. By simulations, we validate its effectiveness in enhancing service coverage. The result will contribute to big data processing in future IoT.