Wireless powered mobile edge computing (MEC) networks, where wireless devices (WDs) are allowed to offload parts of computation-intensive tasks to remote servers and charge the built-in batteries over the air, have been envisaged as a promising technology to ensure the ultra-low-power requirement and enhance the continuous work capacity of WDs. However, when multiple WDs coexist in the network, it is non-trivial to minimize the total tasks delay since the optimization variables are intrinsically coupled. Even more, channels are dynamically varying from time to time and the tasks are unpredictable, which aggravates the difficulty to obtain the closed-form solution. Although reinforcement learning (RL) has been proved to be effective for such complex optimization problems, there is still the challenge that the training of neural networks is time-consuming. This paper considers a challenging hybrid tasks offloading scenario, where offloading tasks can be partially executed locally and remotely in parallel, and each WD is endowed to take both the active RF-transmission and passive backscatter communication (BackCom) for remote tasks offloading. Furthermore, a game-combined multi-agent deep deterministic policy gradient (MADDPG) algorithm is proposed to minimize the total tasks delay with the fairness consideration of multiple WDs, i.e., potential game for offloading decision and MADDPG for time scheduling and harvested energy splitting. The introduction of potential game which can be proved to converge with finite iterations, helps to accelerate the training and reduce the computation complexity. Equipped with the feature of ‘centralized training with decentralized execution,’ once well trained, each agent in MADDPG can figure out the proper time scheduling and harvested energy splitting independently without sharing information with others. Besides the unilateral contention among WDs for the offloading decision by potential game, a fully decentralized framework is finally designed for the proposed algorithm. Numerical results demonstrate that the game-combined MADDPG algorithm can achieve the near-optimal performance compared with existing centralized approaches, and reduce the convergence time compared with other no-game learning approaches.

Cooperative spectrum sensing (CSS) technology has been widely studied to enhance the spectrum sharing efficiency spatially and temporally in cognitive radio networks (CRNs), where the secondary users (SUs) can opportunistically reuse the channels already licensed to the primary users (PUs) for transmission by sensing spectrum holes. SUs are endowed with the global awareness of channels state by cooperating with each other without sweeping across the whole frequency bands. Since the channels occupation of PUs changes dynamically, the accurate sensing and swift information sharing are crucial for CRNs. The paper proposes a multi-agent deep reinforcement learning (DRL) based CSS method to help SUs efficiently finding a vacant channel by the cooperation with their partners. 1 Two partner selection algorithms are proposed named as Reliable Partner CSS and Adaptive Partner CSS, respectively. For the former, the partner selection is facilitated based on the historical sensing accuracy of SUs. While the latter takes the comprehensive consideration of both the reliability and geographical distribution of SUs to further improve the sensing accuracy. 2 Multi-agent deep deterministic policy gradient (MADDPG) is adopted to resist the dynamically varying channels condition as well as the high-dimension solution space. With the feature of ‘centralized training and decentralized execution’, each SU learns to interact with the environment and select a vacant channel for transmission by its partial observation, which greatly reduces the communication overhead caused by the cooperative spectrum sensing. 3 Numerical simulation demonstrates the convergence and availability of the proposed algorithms. No matter Reliable Partner CSS or Adaptive Partner CSS, the sensing accuracy can be greatly enhanced comparing with other non-cooperative or centralized learning approaches. Besides, the attention mechanism is introduced to MADDPG for Adaptive Partner CSS to reveal the behavior of SUs by the visualization of attention weight, which helps to partially interpret the ‘black box’ issue of DRL.

Due to the high flexibility and mobility, unmanned aerial vehicles (UAVs) can be deployed as aerial relays touring to serve ground users (GUs), especially when the ground base station is temporally damaged. However, the broadcasting nature of wireless channels makes such communication vulnerable to be wiretapped by malicious eavesdropping users (EUs). Besides the collecting offloading data for legitimate GUs, UAVs are also expected to be friendly jammers, i.e., generating artificial noise (AN) to deteriorate the wiretapping of EUs. With this in mind, a novel role switching scheme (RSS) is proposed in the paper to guarantee the secure communication by the cooperation of multiple UAVs, where each UAV is allowed to switch its role as a collector or a jammer autonomously to explore a wider trajectory space. It’s worthy to be noticed that the joint optimization for the trajectory of UAVs and the transmission power of GUs and UAVs with role switching scheme is a non-convex mixed integer non-linear programming (MINLP) problem. Since the relaxation of binary variables will lead the solution dropping into local minimum, a deep reinforcement learning (DRL) combined successive convex approximate (SCA) algorithm is further designed to maximize the achievable secrecy rate (ASR) of GUs. Numerical results illustrate that compared with the role fixed scheme (RFS) and relaxation based SCA approaches, the proposed DRL-SCA algorithm endows UAVs the capacity to fly close enough to target users (both GUs and EUs) with less moving distance which brings better ASR and less energy consumption.

The paper considers a more challenging task offloading scenario in hybrid UAV-assisted mobile edge computing (MEC) systems, where multiple dual-function UAVs tour in the sky to serve ground users (GUs) by acting as edge servers or aerial relays. Since each task can be executed on GUs, UAVs and the base station (BS) in parallel, the service assignment, task splitting, trajectory of UAVs, as well as resource and transmission power of both UAVs and GUs should be jointly optimized to minimize the system energy consumption with the subjection of the maximum tolerable latency and computing limitations. To tackle such mixed integer non-linear programming (MINLP) problem, a deep reinforcement learning (DRL) combined successive convex approximation (SCA) algorithm is proposed in the paper to seek a close optimal solution with low-complexity. In specific, the binary service assignment and continuous task splitting are obtained by DRL, while the trajectory planning and resource scheduling are jointly optimized by SCA in sequence to speed up the convergence. Numerical results demonstrate that the proposed DRL-SCA algorithm equipped with dual-function UAV scheme is more effective in making full use of the on-board resource of UAVs and reducing the overall system energy consumption.

To combat the spectrum scarcity, non-orthogonal multiple access (NOMA) and vehicle-to-everything (V2X) systems are integrated as NOMA-V2X networks, where multiple vehicleto-vehicle (V2V) links can opportunistically reuse the spectrum licensed to vehicle-to-infrastructure (V2I) links. However, the contradictory quality of service (QoS) requirements among V2V and V2I links make the design of an effective spectrum sharing scheme to be a great challenge. The high mobility of vehicles and the extra interference brought by NOMA make the issue more complex. Hence, a matching combined multi-agent deep deterministic policy gradient (MADDPG) algorithm is proposed in the paper to maximize the sum delivery rate of V2I up-links while guaranteeing the reliability of V2V links in NOMA-V2X networks. In specific, the channel assignment is solved by oneto-many matching in advance which can be theoretically proved to converge to a stable state. While heterogeneous MADDPG is further adopted to obtain the proper power control for V2I link pairs and V2V links which are taken as different types of agents and interact with the environment independently. On the basis, a fully decentralized framework is designed for the proposed algorithm to reduce the communication overhead caused by the information synchronization. Simulation results demonstrate that by introducing matching theory into deep reinforcement learning (DRL), the sum delivery rate of V2I links can be greatly improved with less computation complexity and convergence time. Moreover, compared with orthogonal multiple access (OMA) communications, the outstanding energy and spectrum efficiency make it significant to explore NOMA in V2X networks

We introduce herein the induced-charge electrokinetic phenomenon to nanometer fluidic systems; the design of the nanofluidic ion diode for field-effect ionic current control of the nanometer dimension is developed by enhancing internal ion concentration polarization through electrochemical transport of inhomogeneous inducing-counterions resulting from double gate terminals mounted on top of a thin dielectric layer, which covers the nanochannel connected to microfluidic reservoirs on both sides. A mathematical model based on the fully-coupled Poisson-Nernst-Plank-Navier-Stokes equations is developed to study the feasibility of this structural configuration causing effective ionic current rectification. The effect of various physiochemical and geometrical parameters, such as the native surface charge density on the nanochannel sidewalls, the number of gate electrodes (GE), the gate voltage magnitude, and the solution conductivity, permittivity, and thickness of the dielectric coating, as well as the size and position of the GE pair of opposite gate polarity, on the resulted rectification performance of the presented nanoscale ionic device is numerically analyzed by using a commercial software package, COMSOL Multiphysics (version 5.2). Three types of electrohydrodynamic flow, including electroosmosis of 1st kind, induced-charge electroosmosis, and electroosmosis of 2nd kind that were originated by the Coulomb force within three distinct charge layers coexist in the micro/nanofluidic hybrid network and are shown to simultaneously influence the output current flux in a complex manner. The rectification factor of a contrast between the ‘on’ and ‘off’ working states can even exceed one thousand-fold in the case of choosing a suitable combination of several key parameters. Our demonstration of field-effect-tunable nanofluidic ion diodes of double external gate electrodes proves invaluable for the construction of a flexible electrokinetic platform for ionic current control and may help transform the field of smart, on-chip, integrated circuits.