Vehicular edge computing has been a promising paradigm to offer low-latency and high reliability vehicular services for users. Nevertheless, for compute-intensive vehicle applications, most previous researches cannot perform them efficiently due to both the inadequate of infrastructure construction and the computing resource bottleneck of the edge server. Motivated by the fact that there is a large number of outside parked vehicles with rich and underutilized resources in the urban area, we propose the idea of parking edge computing, which makes use of the parked vehicles to assist edge servers in offloaded task handling. Specifically, on-street and off-street parked vehicles are first organized into parking clusters to act as virtual edge servers, participating in offloaded tasks execution in our framework. Second, a novel task scheduling algorithm is designed to jointly decide edge server selection and resource assignment. Furthermore, a local task scheduling policy is proposed as well, which reasonably allocates parked vehicles to perform the tasks with the aim of further improving task offloading performance. Finally, a time-related trajectory prediction model based on the random forest model is built, which helps to send back output result accurately. Our framework not only requires no additional infrastructure investment but also provides adequate computing resources. Simulation results based on a real city map and realistic traffic situations demonstrate that our framework provides more efficient and stable offloading services, especially in a large number of task requests condition.

Energy problem is one of the most important challenges in the application of sensor networks. With the development of wireless charging technology and intelligent mobile charger (MC), the energy problem can be solved by wireless charging strategy. In the practical application of Wireless Rechargeable Sensor Networks (WRSNs), the energy consumption rate of nodes is dynamically changed due to many uncertainties such as the death and different transmission tasks of sensor nodes. However, existing works focus on on-demand schemes, which not fully consider real-time global charging scheduling. In this paper, a novel Dynamic Charging Scheme (DCS) in WRSN based on Actor-Critic Reinforcement Learning (ACRL) algorithm is proposed. In the ACRL, we introduce Gated Recurrent Units (GRU) to capture the relationships of charging actions in time sequence. Using the Actor network with one GRU layer, we can pick up an optimal or near-optimal sensor node from candidates as the next charging target more quickly and speed up the training of the model. Meanwhile, we take the tour length and the number of dead nodes as the reward signal. Actor and Critic networks are updated by the error criterion function of R and V. Compared with current on-demand charging scheduling algorithms, extensive simulations show that the proposed ACRL algorithm surpasses heuristic algorithms such as the Greedy, DP, NJNP and TSCA in the average lifetime and tour length, especially against the size and complexity increasing of WRSNs.

Since pervasive smartphones own advanced computing capability and are equipped with various sensors, they have been used for dangerous driving behaviors detection, such as drunk driving. However, sensory data gathered by smartphones are noisy, which results in inaccurate driving behaviors estimations. Some existing works try to filter noise from sensor readings, but usually only the outlier data are filtered. The noises caused by hardware of the smartphone cannot be removed from the sensor reading. In this paper, we propose DrivingSense, a reliable dangerous driving behavior identification scheme based on smartphone autocalibration. We first theoretically analyze the impact of the sensor error on the vehicle driving behavior estimation. Then, we propose a smartphone autocalibration algorithm based on sensor noise distribution determination when a vehicle is being driven. DrivingSense leverages the corrected sensor parameters to identify three kinds of dangerous behaviors: speeding, irregular driving direction change, and abnormal speed control. We evaluate the effectiveness of our scheme under realistic environments. The results show that DrivingSense, on average, is able to detect the driving direction change event and abnormal speed control event with 93.95% precision and 90.54% recall, respectively. In addition, the speed estimation error is less than 2.1 m/s, which is an acceptable range.

With the emergence of wireless rechargeable sensor networks (WRSNs), the possibility of wirelessly recharging nodes using mobile charging vehicles (MCVs) has become a reality. However, existing approaches overlook the effective integration of node energy replenishment and mobile data collection processes. In this paper, we propose a joint energy replenishment and data collection scheme (D-JERDG) for WRSNs based on deep reinforcement learning. By capitalizing on the high mobility of unmanned aerial vehicles (UAVs), D-JERDG enables continuous visits to the cluster head nodes in each cluster, facilitating data collection and range-based charging. First, D-JERDG utilizes the K-means algorithm to partition the network into multiple clusters, and a cluster head selection algorithm is proposed based on an improved dynamic routing protocol, which elects cluster head nodes based on the remaining energy and geographical location of the cluster member nodes. Afterward, the simulated annealing (SA) algorithm determines the shortest flight path. Subsequently, the DRL model multiobjective deep deterministic policy gradient (MODDPG) is employed to control and optimize the UAV instantaneous heading and speed, effectively planning UAV hover points. By redesigning the reward function, joint optimization of multiple objectives such as node death rate, UAV throughput, and average flight energy consumption is achieved. Extensive simulation results show that the proposed D-JERDG achieves joint optimization of multiple objectives and exhibits significant advantages over the baseline in terms of throughput, time utilization, and charging cost, among other indicators.

Wireless rechargeable sensor networks (WRSN) have been emerging as an effective solution to the energy constraint problem of wireless sensor networks (WSN). However, most of the existing charging schemes use Mobile Charging (MC) to charge nodes one-to-one and do not optimize MC scheduling from a more comprehensive perspective, leading to difficulties in meeting the huge energy demand of large-scale WSNs; therefore, one-to-multiple charging which can charge multiple nodes simultaneously may be a more reasonable choice. To achieve timely and efficient energy replenishment for large-scale WSN, we propose an online one-to-multiple charging scheme based on Deep Reinforcement Learning, which utilizes Double Dueling DQN (3DQN) to jointly optimize the scheduling of both the charging sequence of MC and the charging amount of nodes. The scheme cellularizes the whole network based on the effective charging distance of MC and uses 3DQN to determine the optimal charging cell sequence with the objective of minimizing dead nodes and adjusting the charging amount of each cell being recharged according to the nodes’ energy demand in the cell, the network survival time, and MC’s residual energy. To obtain better performance and timeliness to adapt to the varying environments, our scheme further utilizes Dueling DQN to improve the stability of training and uses Double DQN to reduce overestimation. Extensive simulation experiments show that our proposed scheme achieves better charging performance compared with several existing typical works, and it has significant advantages in terms of reducing node dead ratio and charging latency.

To meet the huge energy demand of large-scale wireless rechargeable sensor networks (WRSN), using multiple wireless charging vehicles (MCV) is a natural way, however, which also results in high costs of construction and operation. A new separable charging mode is regarded as a feasible and low-cost method to replenish energy for large-scale WRSN. Nonetheless, existing works consider the deployment and recycling of wireless chargers as two completely separate phases, neglect the intrinsic interactions between them, and thus still suffer performance problem. To improve charging performance, we study the dynamic charging-recycling scheduling (DCRS) problem, and prove that it is NP-hard. Then, we propose a deep reinforcement learning based scheme to get a near optimal solution of the DCRS problem. The proposed scheme utilizes double deep Q-learning network to jointly optimize the charging and recycling scheduling of MCV, responding to more charging requests through timely recycling wireless chargers. The Q-network extracts the feature information of WRSN and determines whether and when to charge a sensor node or recycle a wireless charger. The target Q-network optimizes the charging-recycling strategy, which can alleviate the overestimation problem and improve the stability of training. Furthermore, we analyze the minimum number of wireless chargers under the condition of ensuring a WRSN run stably. The simulation results show that the proposed scheme outperforms the state-of-the-art scheme in terms of number of dead nodes and charging delay.