Multi-Agent Reinforcement Learning (MARL) has shown strong advantages in urban multi-intersection traffic signal control, but it also suffers from the problems of non-smooth environment and inter-agent coordination. However, most of the existing research on MARL traffic signal control has focused on designing efficient communication to solve the environment non-smoothness problem, while neglecting the coordination between agents. In order to coordinate among agents, this paper combines MARL and the regional mixed-strategy Nash equilibrium to construct a Deep Convolutional Nash Policy Gradient Traffic Signal Control (DCNPG-TSC) model, which enables agents to perceive the traffic environment in a wider range and achieves effective agent communication and collaboration. Additionally, a Multi-Agent Distributional Nash Policy Gradient (MADNPG) algorithm is proposed in this model, which is the first time the mixed-strategy Nash equilibrium is used for the improvement in the Multi-Agent Deep Deterministic Policy Gradient algorithm traffic signal control strategy to provide the optimal signal phase for each intersection. In addition, the eco-mobility concept is integrated into MARL traffic signal control to reduce pollutant emissions at intersections. Finally, simulation results in synthetic and real-world traffic road networks show that DCNPG-TSC outperforms other state-of-the-art MARL traffic signal control methods in almost all performance metrics, because it can aggregate the information of neighboring agents and optimize the agent’s decisions through gaming to find an optimal joint equilibrium strategy for the traffic road network.
As an important tool in the field of mathematics, higher-order derivation problems are widely used in differentials, quantum mechanics, and engineering applications. However, in the electronic computer (EC), due to the existence of the carry in the calculation, the computational efficiency is low when solving the higher-order derivation problem. In response to this problem, the ternary optical computer (TOC) has the advantages of no carry-in and the characteristics of numerous data bits, reconfigurable processors and parallel computing. Solve the higher-order derivation problems with complex operations by constructing multipliers and adders on the TOC platform, and copying multiple composite operator units (COUs). This article introduces the design of the higher-order derivative algorithm based on TOC in detail, the reconfiguration process of the multiplier and adder, and the number of bits of the multiplier and adder required in the implementation is given. Besides, the hardware resources and clock cycles in the operation are analyzed. The feasibility of the implementation scheme is verified by experiments. Compared with the traditional higher-order derivative, the higher-order derivative based on the TOC is superior in time performance, computational efficiency, and processing of complex operations. Due to the limitation of the research stage, the algorithm is only applicable to the function of polynomials, which lays a foundation for the further research of higher-order derivative algorithms, and has certain application significance.