The vehicular network needs efficient and reliable data communication technology to maintain low latency. It is very challenging to minimize the energy consumption and data communication delay while the vehicle is moving and wireless channels and bandwidth are time-varying. With the help of the emerging mobile edge computing (MEC) server, vehicles and roadside units (RSUs) can offload computing tasks to MEC associated with base station (BS). However, the environment for offloading tasks to MEC, e.g., wireless channel states and available bandwidth, is unstable. Therefore, ensuring the efficiency of computing offloading under such an unstable environment is a challenge. In this work, we design a task computation offloading model in a heterogeneous vehicular network; this model takes into account multiple stochastic tasks, the variety of wireless channels and bandwidth. To obtain the tradeoff between the cost of energy consumption and the cost of data transmission delay and avoid curse of dimensionality caused by the complexity of the large action space, we propose an adaptive computation offloading method based on deep reinforcement learning (ACORL) that can address the continuous action space. ACORL adds the Ornstein-Uhlenbeck (OU) noise vector to the action space with different factors for each action to validate the exploration. Multi transmission equipment can execute local processing and computing offloading to MEC. Nevertheless, ACORL considers the variety of wireless channels and available bandwidth between adjacent time slots. The numerical results illustrate that the ACORL algorithm can effectively learn the optimal policy, which outperforms the Dueling DQN and greedy policy in the stochastic environment.

A large number of connected sensors and devices in Internet of Things (IoT) can generate large amounts of computing data and increase massive energy consumption. Real-time states monitoring and data processing of IoT nodes are of great significance, but the processing power of IoT devices is limited. Using the emerging mobile edge computing (MEC), IoT devices can offload computing tasks to an MEC server associated with small or macro base stations. Moreover, the use of renewable energy harvesting capabilities in base stations or IoT nodes may reduce energy consumption. As wireless channel conditions vary with time and the arrival rates of renewable energy, computing tasks are stochastic, and data offloading and renewable energy aware for IoT devices under a dynamic and unknown environment are major challenges. In this work, we design a data offloading and renewable energy aware model considering an MEC server performing multiple stochastic computing tasks and involving time-varied wireless channels. To optimize data transmission delay, energy consumption, and bandwidth allocation jointly, and to avoid the curse of dimensionality caused by the complexity of the action space, we propose a joint optimization method for data offloading, renewable energy aware, and bandwidth allocation for IoT devices based on deep reinforcement learning (JODRBRL), which can handle the continuous action space. JODRBRL can minimize the total system cost(including data buffer delay cost, energy consumption cost, and bandwidth cost) and obtain an efficient solution by adaptively learning from the dynamic IoT environment. The numerical results demonstrate that JODRBRL can effectively learn the optimal policy, which outperforms Dueling DQN, Double DQN (DDQN), and greedy policy in stochastic environments.

Users in heterogeneous wireless networks may generate massive amounts of data that are delay-sensitive or require computation-intensive processing. Owing to computation ability and battery capacity limitations, wireless users (WUs) cannot easily process such data in a timely manner, and mobile edge computing (MEC) is increasingly being used to resolve this issue. Specifically, data generated by WUs can be offloaded to the MEC server for processing, which has greater computing power than WUs. However, as the location of MEC servers is fixed, unmanned aerial vehicles (UAVs) have been considered a promising solution in heterogeneous wireless networks. In this study, we design an UAV-assisted computation offloading scheme in an MEC framework with renewable power supply. The proposed model considers the instability of energy arrival, stochastic computation tasks generated by WUs, and a time-varying channel state. Owing to the complexity of the state, it is difficult to use traditional Markov decision process (MDP) with complete prior knowledge for offloading optimization. Accordingly, we propose UAV-assisted computation offloading for MEC based on deep reinforcement learning (UACODRL) to minimize the total cost, which is the weighted sum of the delay, energy consumption, and bandwidth cost. We first use the K-Means algorithm for classification to reduce the dimension of the action space. Subsequently, we use UACODRL to find the near-optimal offloading scheme to minimize the total cost. Simulations demonstrate that UACODRL converges satisfactorily and performs better than four baseline schemes with different parameter configurations.

In heterogeneous wireless networks, massive mobile terminals randomly generate a large number of computation tasks (payloads). How to better manage these mobile terminals located in wireless networks to achieve acceptable quality of service (QoS) such as latency minimization, energy consumption minimization is crucial. A multi-access edge computing (MEC) server can be leveraged to execute the offloaded payloads generated from mobile terminals owing to its powerful processing power and location proximity features. However, an MEC server cannot tackle all offloaded tasks from multiple mobile terminals, and its energy consumption needs further consideration. We introduce an edge server model combined with the unmanned aerial vehicle (UAV) and equipped with the macro base station (MBS-MEC) to process the arrival payloads, and all UAVs and MBS-MECs can harvest renewable energy by using energy harvesting equipment. Furthermore, we model the computation offloading as a deep reinforcement learning scheme without priori knowledge. Considering the infeasibility of deep-reinforcement learning-based centralized learning for the proposed edge computing framework, we propose a distributed computation offloading scheme based on deep reinforcement learning (DCODRL) to minimize the weighted average cost, including the latency cost and the energy cost. Each mobile terminal can be regarded as a learning agent for the DCODRL. To compensate for the lack of cooperation of the DCODRL, we propose a gated-recurrent-unit-assisted multi-agent computation offloading scheme based on deep reinforcement learning (MCODRL) to improve the offloading policy by obtaining global observation information and designing a common reward for all agents. Comprehensive numerical results reflect the convergence and effectiveness of the DCODRL and MCODRL, and the efficacy of the proposed algorithms is further verified through comparisons with two baseline algorithms.