The rapid increase in carbon emissions from the logistics transportation industry has underscored the urgent need for low-carbon logistics solutions. Electric logistics vehicles (ELVs) are increasingly being considered as replacements for traditional fuel-powered vehicles to reduce emissions in urban logistics. However, ELVs are typically limited by their battery capacity and load constraints. Additionally, effective scheduling of charging and the management of transportation duration are critical factors that must be addressed. This paper addresses low energy consumption scheduling (LECS) problem, which aims to minimize the total energy consumption of heterogeneous ELVs with varying load and battery capacities, considering the availability of multiple charging stations (CSs). Given that the complexity of LECS problem, this study proposes a heterogeneous attention model based on encoder-decoder architecture (HAMEDA) approach, which employs a heterogeneous graph attention network and introduces a novel decoding procedure to enhance solution quality and learning efficiency during the encoding and decoding phases. Trained via deep reinforcement learning (DRL) in an unsupervised manner, HAMEDA is adept at autonomously deriving optimal transportation routes for each ELV from specific cases presented. Comprehensive simulations have verified that HAMEDA can diminish overall energy utilization by no less than 1.64% compared with other traditional heuristic or learning-based algorithms. Additionally, HAMEDA excels in maintaining an advantageous equilibrium between execution speed and the quality of solutions, rendering it exceptionally apt for expansive tasks that necessitate swift decision-making processes.

With the swift development of express delivery industry, the increasingly attention has been shifted to express delivery mechanism design. Generally, the revenue of the courier is the difference between the users' express fee and the courier's pickup cost. In order to improve the revenue of courier without increasing the user's express fee, this paper presents a low-cost package pickup covering system to find an optimal Hamiltonian pickup tour for the courier over a subset of packages, where packages who are not on the tour should be covered exactly by one package on the tour. A billing rule discounting the express fee to incentivize users to deliver their packages is also proposed. We formulate Low-cost Package Pickup Covering (LPPC) problem to maximize the revenue of the courier. Considering the complexity of LPPC , we propose a Low-cost Package Pickup Covering Mechanism (LPPCM) to solve the LPPC problem including problem transformation, hardness analyzing, Attention Model based on Encoder-Decoder Architecture (AMEDA) model design and model training. AMEDA is trained by a deep reinforcement learning algorithm in an unsupervised manner and it can directly output the solution based on the given instances. Through extensive simulations, we demonstrate that the average revenue of courier for AMEDA is at least 10.1% higher than the traditional heuristic local search and is 18.5% lower than the optimal solution on average. AMEDA provides a desired trade-off between the execution time and solution quality, which is well suited for the large-scale tasks which require quick decisions.

In the mobile edge computing (MEC)-enabled air-terrestrial integrated network (ATIN), uncrewed aerial vehicles (UAVs) serve as air edge nodes with the purpose to collaboratively train the high-availability prediction model by federated learning (FL). Nonetheless, in view of the significance of data freshness for an accurate training model, UAVs suffer from the stochastic and intermittent nature in energy harvesting (EH). This article formulates an inter-UAV cooperative federated computing (IUCFC) problem to jointly optimize prediction accuracy, age of information (AoI) in flight region, and overall energy consumption of EH-enabled UAVs for edge data processing. To address the intricate IUCFC problem, a deep reinforcement learning (DRL)-based cooperative UAV intelligent decision (CUID) algorithm is proposed, which leverages a dual actor–critic architecture, in pursuit of the collective tuning of hybird actions. Furthermore, the Ornstein–Uhlenbeck (OU) noise is engaged in continuous action spaces to prompt exploration, while a conditional iteration dropout (CID) scheme mitigates the infeasible actions caused by the noise introduced, thereby bolstering the exploration efficiency and quality of CUID algorithm. Considering the nonstationary environments originated from UAV mobility, priority experience replay (PER) is adopted to dynamically modify experience priority. Extensive experiments show that CUID attains superior performance over those advanced algorithms, upgrading system utility by 8.79%, while augmenting FL model accuracy by 3.68% in dynamic scenarios with heterogeneous data distributions.

Vehicle-to-everything (V2X) links are crucial to the infrastructure of smart transportation and cities, supporting modern communication technology. However, limited spectrum resources and access conflicts hinder achieving low-latency and high-reliability V2X links. Additionally, rapid channel variations and inter-link interference pose challenges in acquiring global information for centralized resource management in V2X networks. To address these issues, a sub-band allocation and power control algorithm based on multi-agent deep reinforcement learning (DRL) is proposed. Considering the complexity of the inter-link spectrum sharing (ILSS) problem in dynamic vehicular networks, this study proposes a multi-agent adaptive hybrid deep Q-Network (MA2HDQN). The network integrates the multi-agent dueling double deep Q-network (MA-D3QN) with the improved deep deterministic policy gradient network (i-DDPG), optimizing sub-band allocation and power control by segregating discrete and continuous actions within the global action space. To expedite Q-network convergence, an adaptive learning rate update strategy is proposed, ensuring timely Q-value updates in response to recent environmental feedback within the MA-D3QN network. Moreover, a multi-step strategy is utilized to update Q-values, enhancing the overall system utility and stability of the proposed MA2HDQN algorithm. Simulation results demonstrate that the proposed algorithm achieves stable and rapid convergence, significantly improving the capacity of vehicle-to-infrastructure (V2I) links and the reliability of vehicle-to-vehicle (V2V) links.

The Integrated Energy System (IES) integrates energy consumption, conversion, and storage to increase the utilization of renewable energy sources and fossil fuels. When using heuristic algorithms and mathematical optimization methods for optimal scheduling of IES, these methods often face challenges of high computational cost and insufficient model accuracy, for which Reinforcement Learning (RL) offers a promising solution. However, most studies have used Model-Free Reinforcement Learning (MFRL) for optimal scheduling, which is limited by problems of low sampling efficiency and long training time. Model-Based Reinforcement Learning (MBRL) is an effective alternative to accelerate learning and facilitate RL applications. Therefore, this paper compares the performance gap between MFRL and MBRL in the optimal scheduling problem of IES to determine the superiority of MBRL. Experimental results show that the MBPO algorithm significantly improves sampling efficiency and learning speed at high prediction accuracy, outperforming the Soft Actor-Critic (SAC) algorithm.

To address the increasingly severe issue of insufficient vehicular caching in the Internet of Vehicles (IoV), Vehicular edge computing uses base stations (BS) and vehicles with computing and storage capabilities as edge servers to cache content. However, most existing studies focus on reducing task costs, assuming vehicles can simultaneously act as requesters and providers, without considering whether a vehicle is suitable to serve as a provider, willing to share resources or ensure secure data transmission, thus lowering matching efficiency. Deep reinforcement learning and blockchain technology were integrated into the Internet of Vehicles to address distributed vehicle-to-vehicle collaborative caching. An optimal collaborative caching strategy was designed, considering dynamic vehicle attributes, with the system's objective being to maximize vehicle utility to encourage resource sharing. Simulation results confirm that the proposed blockchain-based vehicular edge computing collaborative caching scheme with deep reinforcement learning significantly outperforms other strategies.