This study develops a traffic-responsive control technique for fully-actuated coordinated signals (TCT) to minimize investment in data collection and engineering judgment for signal timing. The operating process of TCT in a day is divided into consecutive cycle sets with $N$ cycles for each. A vehicle detector is placed on every approach lane to detect time headways and actuate the coordinated and uncoordinated phases. Control center creates the background plans for all the intersections once per cycle set. The fully-actuated logic settings are directly determined by the timing parameters of the background plans in signal controllers. Signal controllers operate the fully-actuated logic and yield the expected greens of the 1 ^st through the $(N-1)^{\mathrm{ th}}$ cycles of the current cycle set for the coordinated and uncoordinated phases. After the $N^{\mathrm{ th}}$ cycle starts, the expected greens are sent to control center. Control center processes the expected greens generated in the past cycles of the day and feeds them into Brown’s double exponential smoothing to predict the expected base greens of the next cycle set. They pave the way to stepwise creating the new background plan without using any optimization models. The simulation results indicated that the TCT-enabled intersections could reactively vary their vehicle handling capacity to accommodate the variation in the vehicle demand. In contrast to the conventional control technique for fully-actuated coordinated signals, TCT could provide more progression opportunities to the coordinated movement and serve vehicles with less number of stops and equivalent delay from a system-wide perspective.

This paper proposes modified model predictive control (MMPC) for coordinated signals, aiming to enhance a model’s fidelity to the realistic traffic environment by relaxing typical assumptions. We focus on the arterial, where every intersection is equipped with a dual‐ring‐barrier signal controller that complies with the standards of the National Electric Manufacturers Association. MMPC employs the store‐and‐forward model to describe traffic flow, thereby transforming the signal control problem into a model‐based rolling‐horizon optimization problem, in which the prediction horizon is composed of several future sample intervals, commonly equal to the cycle length. A radar detector is used to collect vehicle data upstream of the stop line at every sampling instant. The optimization problem is solved to minimize the number of vehicles within the prediction horizon, and the next timing plan is determined based on the optimization results. Constraints are added and modified in order to incorporate the typical relaxed assumptions in the optimization process. For this purpose, MMPC introduces a transition‐free ring‐barrier structure, vehicle distribution ratio, and percent arrival before the end of green. Simulation results indicate that coordination can be maintained by MMPC without the need for transitions, and the estimation of current and future traffic states can be improved with the assistance of modified constraints. Compared with benchmark techniques, MMPC offers superior vehicle progression for coordinated movement and significant improvements in delays, number of stops, and total travel time from a system‐wide perspective, with an acceptable small increase in runtime.

Explicit signal coordination carries prior knowledge of traffic engineering and is widely accepted for global implementation. With the recent popularity of reinforcement learning, numerous researchers have turned to implicit signal coordination. However, these methods inevitably require learning coordination from scratch. To maximize the use of prior knowledge, this study proposes an explicit coordinated signal control (ECSC) method using a soft actor–critic for cycle length determination. This method can fundamentally solve the challenges encountered by traditional methods in determining the cycle length. Soft actor–critic was selected among various reinforcement learning methods. A single agent was administered to the arterials. An action is defined as the selection of a cycle length from among the candidates. The state is represented as a feature vector, including the cycle length and features of each leg at every intersection. The reward is defined as departures that indirectly minimize system vehicle delays. Simulation results indicate that ECSC significantly outperforms the baseline methods, as evident in system vehicle delay across nearly all demand scenarios and throughput in high demand scenarios. The ECSC revitalizes explicit signal coordination and introduces new perspectives on the application of reinforcement learning methods in signal coordination.

Real-time isolated signal control (RISC) at an intersection is of interest in the field of traffic engineering. Energizing RISC with reinforcement learning (RL) is feasible and necessary. Previous studies paid less attention to traffic engineering considerations and under-utilized traffic expertise to construct RL tasks. This study profiles the single-ring RISC problem from the perspective of traffic engineers, and improves a prevailing RL method for solving it. By qualitative applicability analysis, we choose double deep Q-network (DDQN) as the basic method. A single agent is deployed for an intersection. Reward is defined with vehicle departures to properly encourage and punish the agent’s behavior. The action is to determine the remaining green time for the current vehicle phase. State is represented in a grid-based mode. To update action values in time-varying environments, we present a temporal-difference algorithm TD(Dyn) to perform dynamic bootstrapping with the variable interval between actions selected. To accelerate training, we propose a data augmentation based on intersection symmetry. Our improved DDQN, termed D3ynQN, is subject to the signal timing constraints in engineering. The experiments at a close-to-reality intersection indicate that, by means of D3ynQN and non-delay-based reward, the agent acquires useful knowledge to significantly outperform a fully-actuated control technique in reducing average vehicle delay.

Model-free deep reinforcement learning (MFDRL) is well-suited for real-time traffic signal control (RTSC), as it is a sequential decision problem where the environment is difficult to be a priori modeled, but has performance metrics sufficing as rewards. Previous studies have not ideally employed MFDRL systems at typical intersections with a dual-ring barrier phase structure (DRBPS) and second-by-second signal operation. DRBPS allows phases to time flexibly while satisfying signal timing constraints in engineering, making it complicated yet common in real-world applications. This study proposes an MFDRL method, termed MD4-AQF, to address the RTSC problem under DRBPS. The state can be represented as a 4 × 30 × 4 × 4 array. We define action based on “decision point aligning” to produce a consistent action space that controls dual-ring concurrent phases simultaneously. We developed a training algorithm based on a “multi-dimensional Q-network” that reduces the number of learnable actions from 600+ to 52. We designed action selection based on “aligned Q-fusion” to end two lagging phases simultaneously with a shared compromise sub-action. In simulation experiments, MD4-AQF trains an agent to improve average vehicle delay from 135 s to 48 s. It surpasses another MFDRL ablated method by 14%, and a fully actuated conventional method by 19%.