Connected cruise control (CCC) refers to a type of advanced driver assistance system combined with wireless vehicle-to-vehicle (V2V) communication technology to improve control stability and driving safety. However, it is urgent to investigate intelligent control algorithms to improve the adaptability of CCC in complex traffic environment. In addition, the effect of communication delays attributed to shared wireless communications on the performance degradation of the intelligent CCC design cannot be ignored. In this study, the design of deep reinforcement learning (DRL) controller for CCC system in high-dynamic traffic scenarios is investigated, which considers both time-varying leading velocity and communication delays. To be more specific, an intelligent CCC algorithm based on deep deterministic policy gradient (DDPG) is developed. According to the training samples obtained from interacting with the traffic environment, the actor network and the critic network are trained to maximize the quadratic reward function determined by state errors and control inputs for generating intelligent control strategies. In particular, the effect of previous control strategies is considered in vehicle dynamics analysis, optimization problem formulation and Markov decision process (MDP) definition to compensate the performance degradation attributed to communication delays. Lastly, the effectiveness and convergence of the proposed DRL-based CCC controller are verified through numerical simulations with various conditions. The superior performance of the proposed algorithm is shown by comparing with existing traditional algorithms and state-of-the-art DRL algorithms.

The development of intelligent transportation systems (ITS) has attracted significant attention to connected and autonomous vehicles (CAVs). It is urgent to investigate multi-CAV intelligent cruise control solutions in mixed traffic environments. In addition, the impact of platoon dynamics and time delays, induced by shared wireless communications, data processing, and actuation cannot be ignored. This article investigates the development of a multi-agent deep reinforcement learning (MADRL) controller tailored for CAVs operating within mixed and dynamic traffic scenarios that involve time delays. Firstly, the error dynamics in the discrete-time domain for each subplatoon is derived by considering the time-varying delays and leading vehicle states, and then the optimal CAV cruise control problem is formulated. Subsequently, the partially observable Markov game (POMG) is used to construct the multi-agent environment, and then a centralized training decentralized execution (CTDE) algorithm framework is proposed based on the multi-agent deep deterministic policy gradient (MADDPG) method. Finally, the computational complexity and the influence of delay are analyzed. The simulation results illustrate the effectiveness of the proposed intelligent algorithm.

This paper presents an optimal control algorithm for connected cruise control (CCC) system in the presence of the stochastic communication delays. In the CCC system, wireless vehicle-to-vehicle (V2V) communication is the key factor to achieve comprehensive environment perception while it gives rise to delays in the meantime. In order to compensate for the influence of the stochastic delays and develop an optimal controller for the CCC system, we first analyze the vehicle dynamics with the car-following model. Then, the mathematical platoon model is formulated with heterogenous platoon structure. With the objective of minimizing the deviations of vehicle's headway and velocity, a linear quadratic optimization problem is formulated. The optimal control strategy is solved in an iterative manner by using a backward recursion which can be divided into two steps. In particular, the optimal control gain is iteratively derived off-line and the control strategy is calculated on-line. In addition, the control gain convergence, the stability analysis, and the effect of the packet losses and state noises are analyzed. Finally, the stability and performance of the proposed control algorithm are verified by numerical simulations to highlight its superiority compared with the existing algorithms.

The recent advancements of mobile edge computing (MEC) technologies and unmanned aerial vehicles (UAVs) have provided resilient and flexible computation services for ground users beyond the coverage of terrestrial service. In this paper, we focus on a UAV-assisted MEC system in which the UAV equipped with MEC servers is used to assist user devices in computing their tasks. To minimize the weighted average energy consumption and delay in the UAV-assisted MEC system, a LQR-Lagrange-based DDPG (LLDDPG) algorithm, which jointly optimizes the user task offloading and the UAV trajectory design, is proposed. To be specific, the LLDDPG algorithm consists of three subproblems. The DDPG algorithm is used to address the issue of UAV desired trajectory planning, and subsequently, the LQR-based algorithm is employed to achieve the real-time tracking control of UAV desired trajectory. Finally, the Lagrange duality method is proposed to solve the optimization problem of computational resource allocation. Simulation results indicate that the proposed LLDDPG algorithm can effectively improve the system resource management and realize the real-time UAV trajectory design.

The trajectory design for unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) networks has become a hot research topic. In the UAV-assisted MEC scenario, the UAV is required to frequently adjust its flight trajectory due to dynamic factors such as time-varying task offloading requirements, user mobility, and transmission environment variation. In this paper, with consideration of the constraint induced by the UAV flight dynamics, the dynamic trajectory design challenge within the blockchain-based multi-UAV-assisted MEC framework is investigated. An intelligent algorithm that integrates multi-agent deep deterministic policy gradient (MADDPG), linear quadratic regulator (LQR), and CVXPY solver, named MADDPG-LC, is proposed to achieve real-time joint optimization of dynamic trajectory control and resource allocation with respect to minimizing weighted energy consumption and delays. Numerical simulation results demonstrate the efficacy of the proposed MADDPG-LC algorithm in addressing the UAV flight dynamics constraint, which has generally overlooked in existing works