Internet of Things (IoT) technology suffers from the challenge that rare wireless network resources are difficult to meet the influx of a huge number of terminal devices. Cache-enabled Device-to-Device (D2D) communication technology is expected to relieve network pressure with the facts that the requesting contents can be easily obtained from nearby users. However, how to design an effective caching policy becomes very challenging due to the limited content storage capacity and the uncertainty of user mobility pattern. In this paper, we study the jointly cache content placement and delivery policy for the cache-enabled D2D networks. Specifically, two potential recurrent neural network approaches (the echo state network (ESN) and long-short term memory (LSTM) network) are employed to predict users’ mobility and content popularity, so as to determine which content to cache and where to cache. When the local cache of the user cannot satisfy its own request, the user may consider establishing a D2D link with the neighboring user to implement the content delivery. In order to decide which user will be selected to establish the D2D link, we propose the novel schemes based on deep reinforcement learning to implement the dynamic decision-making and optimization of the content delivery problems, aiming at improving the quality of experience of overall caching system. The simulation results suggest that the cache hit ratio of the system can be well improved by the proposed content placement strategy, and the proposed content delivery approaches can effectively reduce the request content delivery delay and energy consumption.

The Internet of Things (IoT) technology has recently drawn more attention due to its ability to achieve the interconnections of massive physic devices. However, how to provide a reliable power supply to energy-constrained devices and improve the energy efficiency in the wireless powered IoT (WP-IoT) is a twofold challenge. In this paper, we develop a novel wireless power transmission (WPT) system, where an unmanned aerial vehicle (UAV) equipped with radio frequency energy transmitter charges the IoT devices. A machine learning framework of echo state networks (ESNs) together with an improved k-means clustering algorithm is used to predict the energy consumption and cluster all the sensor nodes at the next period, thus automatically determining the charging strategy. The energy obtained from the UAV by WPT supports the IoT devices to communicate with each other. In order to improve the energy efficiency of WP-IoT system, the interference mitigation problem is modeled as a mean field game (MFG), where an optimal power control policy is presented to adapt and analyze the large number of sensor nodes randomly deployed in WP-IoT. The numerical results verify that our proposed dynamic charging policy effectively reduce the data packet loss rate, and that the optimal power control policy greatly mitigate the interference, and improve the energy efficiency of the whole network.

Distributed formation control is presented for networked Euler–Lagrange systems (ELSs) over a directed interaction topology. This problem is defined by a layered framework in which information flow both among the leaders and among the followers is described by different layers. To empower the formation to make a variety of geometric transformations, we present the necessary and sufficient conditions for affine maneuverability under a directed graph. Unlike most existing results using a diagonal stabilizing matrix to achieve the stabilizability of affine formation, this fully distributed approach is feasible without any global information. Next, we propose an adaptive control law for agents in each layer, where the closed-loop errors are driven to a neighborhood of the origin in finite time. Adaptive neural networks are integrated to tackle the model uncertainties in ELSs by updating the norm of the weight matrix, which can simplify the control design and alleviate the computational burden compared with traditional ones. The simulation results are given to show the effectiveness of the proposed approach.

This paper develops a completely distributed affine formation maneuver control (CDAFC) algorithm for networked marine surface vehicles (NMSVs). The desired trajectories are generated by the affine image. Accordingly, NMSVs are capable to achieve arbitrary preset configuration transformations, while only a small part of the formation members called leaders obtain the configuration information. Then, a synthetical constraint is designed, which can concern position and attitude configuration rigidities simultaneously. This feature enables NMSVs to achieve formation maneuvers with time-varying formation speeds and orientations. On these bases, a novel CDAFC scheme is developed. In contrast to the conventional fully distributed control schemes, the implementation requirements are completely limited to the local reference frame, i.e., not only is CDAFC designed in a node-based way, but the CDAFC implementation just depends on the local relative poses and self velocities. Thus, we call it completely distributed. The semi-physical simulations are conducted to illustrate the effectiveness.

To solve the maneuvering decision problem in air combat of unmanned combat aircraft vehicles (UCAVs), in this paper, an autonomous maneuver decision method is proposed for a UCAV based on deep reinforcement learning. Firstly, the UCAV flight maneuver model and maneuver library of both opposing sides are established. Then, considering the different state transition effects of various actions when the pitch angles of the UCAVs are different, the 10 state variables including the pitch angle, are taken as the state space. Combined with the air combat situation threat assessment index model, a two-layer reward mechanism combining internal reward and sparse reward is designed as the evaluation basis of reinforcement learning. Then, the neural network model of the full connection layer is built according to an Asynchronous Advantage Actor–Critic (A3C) algorithm. In the way of multi-threading, our UCAV keeps interactively learning with the environment to train the model and gradually learns the optimal air combat maneuver countermeasure strategy, and guides our UCAV to conduct action selection. The algorithm reduces the correlation between samples through multi-threading asynchronous learning. Finally, the effectiveness and feasibility of the method are verified in three different air combat scenarios.