Past few years have witnessed the widespread adoption of unmanned aerial vehicles (UAVs) in target tracking for regional monitor and strike. Most existing target tracking approaches rely on the target motion frames obtained by the camera equipped, or on ideally assuming a pre-set target trajectory. However, in practice, the real trajectory of the target cannot be perfectly known to the UAVs in advance, and also the target may intelligently adjust its flying strategy according to the environment. Besides, the limited flight performance, as well as information capture and processing capability, of a single UAV can hardly fulfill high tracking success rate requirements. To address aforementioned issues, this paper proposes an end-to-end cooperative multi-agent reinforcement learning (MARL) scheme, where UAVs are enabled to make intelligent flight decisions for cooperative target tracking, on the basis of the past and current states of the target. In order to reduce power consumption and prolong the lifetime of the UAV tracking system, the propulsion power consumption model and energy saving strategy are introduced. Moreover, to further increase the detection coverage, spatial information entropy is introduced in the tracking algorithm. Simulation results show that our proposed algorithm outperfoms the deep reinforcement learning baselines in terms of the mean episode rewards, while also yields high performances with respect to tracking success rates, power saving efficiency and detection coverage.

Vehicular fog computing (VFC) has been expected as a promising scheme that can increase the computational capability of vehicles without relying on servers. Comparing with accessing the remote cloud, VFC is suitable for delay-sensitive tasks because of its low-latency vehicle-to-vehicle (V2V) transmission. However, due to the dynamic vehicular environment, how to motivate vehicles to share their idle computing resource while simultaneously evaluating the service availability of vehicles in terms of vehicle mobility and vehicular computational capability in heterogeneous vehicular networks is a main challenge. Meanwhile, tasks with different priorities of a vehicle should be processed with different efficiencies. In this work, we propose a task offloading scheme in the context of VFC, where vehicles are incentivized to share their idle computing resource by dynamic pricing, which comprehensively considers the mobility of vehicles, the task priority, and the service availability of vehicles. Given that the policy of task offloading depends on the state of the dynamic vehicular environment, we formulate the task offloading problem as a Markov decision process (MDP) aiming at maximizing the mean latency-aware utility of tasks in a period. To solve this problem, we develop a soft actor-critic (SAC) based deep reinforcement learning (DRL) algorithm for the sake of maximizing both the expected reward and the entropy of policy. Finally, extensive simulation results validate the effectiveness and superiority of our proposed scheme benchmarked with traditional algorithms.

Recently, the boosting growth of computation-heavy applications raises great challenges for the Fifth Generation (5G) and future wireless networks. As responding, the hybrid edge and cloud computing (ECC) system has been expected as a promising solution to handle the increasing computational applications with low-latency and on-demand services of computation offloading, which requires new computing resource sharing and access control technology paradigms. This work establishes a software-defined networking (SDN) based architecture for edge/cloud computing services in 5G heterogeneous networks (HetNets), which can support efficient and on-demand computing resource management to optimize resource utilization and satisfy the time-varying computational tasks uploaded by user devices. In addition, resulting from the information incompleteness, we design an evolutionary game based service selection for users, which can model the replicator dynamics of service subscription. Based on this dynamic access model, a Stackelberg differential game based cloud computing resource sharing mechanism is proposed to facilitate the resource trading between the cloud computing service provider (CCP) and different edge computing service providers (ECPs). Then we derive the optimal pricing and allocation strategies of cloud computing resource based on the replicator dynamics of users’ service selection. These strategies can promise the maximum integral utilities to all computing service providers (CPs), meanwhile the user distribution can reach the evolutionary stable state at this Stackelberg equilibrium. Furthermore, simulation results validate the performance of the designed resource sharing mechanism, and reveal the convergence and equilibrium states of user selection, and computing resource pricing and allocation.

To satisfy the expected plethora of computation-heavy applications, federated edge learning (FEEL) is a new paradigm featuring distributed learning to carry the capacities of low-latency and privacy-preserving. To further improve the efficiency of wireless data aggregation and model learning, over-the-air computation (AirComp) is emerging as a promising solution by using the superposition characteristics of wireless channels. However, the fading and noise of wireless channels can cause aggregate distortions in AirComp enabled federated learning. In addition, the quality of collected data and energy consumption of edge devices may also impact the accuracy and efficiency of model aggregation as well as convergence. To solve these problems, this work proposes a dynamic device scheduling mechanism, which can select qualified edge devices to transmit their local models with a proper power control policy so as to participate the model training at the server in federated learning via AirComp. In this mechanism, the data importance is measured by the gradient of local model parameter, channel condition and energy consumption of the device jointly. In particular, to fully use distributed datasets and accelerate the convergence rate of federated learning, the local updates of unselected devices are also retained and accumulated for future potential transmission, instead of being discarded directly. Furthermore, the Lyapunov drift-plus-penalty optimization problem is formulated for searching the optimal device selection strategy. Simulation results validate that the proposed scheduling mechanism can achieve higher test accuracy and faster convergence rate, and is robust against different channel conditions.

In the face of deeply exploring and exploiting marine resources, the Internet of Underwater Things (IoUT) networks have drawn great attention considering its widely distributed low-cost and easy-deployment smart sensing nodes. However, given the hostile underwater environment, it is critical to conceive energy-efficient information collection because of limited underwater energy supply and inefficient artificial recharge methods. Characterized by high flexibility and maneuverability, autonomous underwater vehicles (AUVs) are regarded as a promising solution for information collection in the IoUT relying upon delicate AUVs’ trajectory and information collection strategy design with the spirit of balancing their energy consumption and information processing capability. In this article, we propose a heterogeneous AUV-aided information collection system with the aim of maximizing the energy efficiency of IoUT nodes taking into account AUV trajectory, resource allocation, and the Age of Information (AoI). Moreover, based on the particle swarm optimization (PSO), we obtain the trajectory of AUVs with low time complexity. Additionally, a two-stage joint optimization algorithm based on the Lyapunov optimization is constructed to strike a tradeoff between energy efficiency and system queue backlog iteratively. Finally, simulation results validate the effectiveness and superiority of our proposed strategy.